Hi, I have never been in this situation, but I have watched some close friends go through it, and I have some ideas.
First of all, at 3 your son is probably learning to talk, right? Do you ask him any questions about his time at his dad's, what he likes, what he doesn't like, etc. If it seems like he likes her then I would try to make the most of that relationship, especially being a single mom I'm sure you can use all the help you can get. (I'm not a single mom and I could use more help!)
You said the girlfriend seems like a good person ... how well do you know her? Do you two get along ok? Does she have any kids of her own? My suggestion would be to get to know her even better. Invite her to go play at the park with you guys sometime or some other activity with just the 2 of you and your son (I bet your son would love that).
Before I had kids I used to love spending time with my friends kids, she may be the same way. You never know, she might even be willing to babysit for you when the dad was busy. It sounds like she came into the picture around the same time as your son, so I'm sure there was probably a lot of hurt and anger involved, but hopefully with time you have been able to work through that. I'm not saying you guys have to be best friends, but since she has been with the dad for 3 years she may really enjoy her time with your son, and if you guys knew each other better she might even be easier to talk to about how you would like your son cared for when you aren't around than the dad is, who knows. Best of luck!