All the things that Parimal said led Bepin Babu to think that he had lost the memory of a particular time from the past. Since Bepin Babu’s brother was insane, he feared that this might be the beginning of his going insane too. That is why Bepin Babu worried about what Parimal Ghose said.