C., this will work itself out as kids get older; they clearly will start saying they don't want siblings along and they don't want the parents/grandparents to stay!
Parents tend to stay around in the younger years; or if they don't know you well or haven't met you previously; or if it's a first or maybe second play date. Otherwise they don't often hang around unless they're also friends of the adults and it's a grown-up "play date" too -- which is nice.
Siblings? Our daughter's friends' parents would have always asked if they could bring one, but they never did. Mostly parents seem to see play dates as a good opportunity for some one on one time with the sibling!
I would try, next time with this family or with any other:
"Sally would like to have Jen over to play. Can you drop her off at 2 and pick her up at 4? I'll have a snack for them -- does Jen like Goldfish and milk, or would she prefer graham crackers?" That makes it clear up front that: It's a drop off; it's two hours; you will have a snack, not a meal. Just set out the parameters when you issue the invitation.
If you do want to get to know the parent or haven't really met:
"Sally talks about Jen a lot and would like to have her to our house to play. (Arrange the time and date, and then say) Since she hasn't been here before, you can either drop her off here if you like and have other things to do, or feel free to stay for this play date." If the parent says, I'll have a sibling with me, that makes it tougher but for this one play date you could say, "Sure, bring Sib along," or "If it would be easier for you to have some one on one time with Sib, that's fine, just drop Jen off and we'll do great!"
In other words set out the limits as you go. Length of playdates at age 4, maybe two hours. Siblings brought without asking -- well, not a lot you can do short of saying "Please go" but that's not really tenable. You will find most parents won't do that. I think maybe grandma is just of a different generation and so has different expectations of play date and would cut her slack for that!