When is it OK to say no? All the time. Any time.
If someone truly needs help in a real jam -- sickness in the family so a parent has to take kid one off to the doctor and needs to get kid two a ride somewhere else, for instance -- then of course you want to help and should, if you possibly can.
Everything else you describe sounds truly like it's inconsiderate of others to ask it of you. Kids across town and therefore utterly not even near your own routes? "Sorry, that won't work for us." Kinder kid being dropped off at your house for what is actually an hour of babysitting before school? "Sorry, that won't work for us." Three boys at two locations etc.? "Sorry, that won't...." You get it. If people ask why, first, they have no right to do so, but you can sincerely say, "Our family's stretched to the max" on transportation for your own kids. It seems as if you're reluctant to say those words but they are the truth, and any parent who asks to know more is -- to be really frank -- clueless about how they're using you.
I would just tell parents who are using you for regular rides that as of day X the rides end. "We've looked at our schedules and it's just not working for us to take Jenny to school in the mornings/pick Danny up after scouts and bring him home/whatever. Sorry about that, but we're redoing our runs, and I can continue this week and next but as of May 18 we can't give him/her/them rides to/from any longer."
Be cheery, be firm, but don't cave on the ending date of anything that's become a regular run and which you want to end.
Like many, many women and moms, you likely do just want to help other people and want to be nice and say yes whenever you possibly can, but saying no to transporting others' kids is fine and does not make you a bad or unhelpful person. Put your family first. Don't make excuses for why, other than it's not working for you and you're stretched. Anyone who pesters you for more reasons or tries to tell you that they just can't manage unless you do what you're doing -- they're not actual friends, and they have come to take you for granted. And I'm not saying THEY are bad people, just that they have become thoughtless because you kept saying yes and they assumed you had nothing else to do.
You can make this happen if you can paste on a smile and be firm and NOT get drawn into conversations or negotiations about "Could you give Sally a ride to X just that one week--?"