This line indicates two incidents

a. The sea holiday that her mother had experienced is her past.

b. The fact that the poetess had her mother beside her physically is her past.

So, both of them have lost something. The poetess has lost her mother whereas back in her days, her mother had lost her childhood. Both were not able to believe that the moment of happiness had already slipped out of their hands and were over. That is why both of them were struggling(laboring) to come to terms with the fact that these moments are now present only in the memories.