Borrow money

Mr. J is the cleaner supervisor, he had been working in my working place for few month. Within this few month, I had lent him some money or gave him some money. Until now he still never return me any money. Actually I also never chase him because I know he has financial difficulty.
But I am wondering how come he always like to borrow money from me whenever he see me.
The first time Mr. J told me that he had no money to celebrate his three kids' birthday, I then gave him 20 dollars so that Mr. J could buy birthday cake for his kids.
Other time He told me that he was unable to pay up the renting fees, I gave him 30 dollars. Besides that he always asked for coins, some cash for top up his bus fare card or he needed money for food.
Do you think I should continue lend him money?