K.K.
You've gotten some interesting advice. I'm honestly not sure what i would do either. On one hand, kids do change friends alot, but on the other it is nice to reciprocate unless the mom's are really really laid back.
Just something to think about. Instead of your son inviting the kid(s) over to your house for free play, plan a specific morning out activity or something specific at your house. He might not like just running around with the kid because the kid is too rough, but maybe they could see a matinee on a sat or go to a special story time at the libary type thing, or a nature presentation somewhere, I think Lowes does woodworking things for kids that are reasonable and focused. Or at home, buy special craft kids and invite the child to make a craft with you, or bake something, I guess what i'm getting at is structure, not necessarily something quiet, but something focused, that your son doesn't usually get to do with the friends he really likes and that he would be willing to do with the school friend just because he likes the activity.
I don't know if that helps or not.
If you see his teacher you might ask her if your son really plays with this kid and if she would encourage out of school play dates. if her reaction is a little off this kid might not be the best match for your son. and then i would just let it go.