I haven't dealt with this but I think it would be nice to have the siblings and grandparents there and paying for the grandparent's hotel room, if monetarily feasible, would be a nice gesture. Anyone over and above the grandparents, I can't imagine that anyone would expect you to pay for their room, unless you are able to afford such a generous gesture.
Since it seems that there are so many family members who would like to share in this momentous occasion, perhaps it would be more practical and cost-effective to bring all these people together and have a party for your son, back in your hometown, weeks later or even sometime in the Spring, when the weather may be slightly more predictable. The college graduation may just be a whirlwind weekend and you may be disappointed at how little you see him. Or you can host one over the Holiday season, since he's graduating in December, but... whew... that just may be a lot of work! Hosting the party in your home or vicinity and letting him invite his college friends for a weekend may give him (and you) a chance to enjoy his guests in a more intimate, relaxed setting - especially if you hire a few extra hands to help with the meal preparation, serving, etc...
Congratulations! This is a very exciting time for you. Good luck with whatever you do.