Hi A.,
I've been babysitting 4 children in France, 3, 5, 7 $ 9 years old.
Because they were all day with me, they became so attached to me that in the morning (I was live in au pair) they would call me, not their mom. They would come straight to my room, not downstairs to see the parents. During the night same thing (I wasn't in charge of them night time), but they would call me for anything. BUT, as soon as the mom started going less hours at work and when she was at home more than before, even thought I was still there, living with them, they turned to their mom.
So, no worries, the children are like that, who is most of the time with them, and who is really nice with them, they attach and love that person.
I would suggest spending all your free time with your son, the most you can, and don't worry he will turn to you!
All the best.