In both the cases, the reference is to the giant’s garden. On the first occasion, the children were playing into the garden and were referring to the pleasure they derived out of it. In the second case, the children were not being allowed to play in the garden and were referring to the happiness they derived while they played there in the past.