The level of service you want sounds more like a nanny than a "babysitter." The expectation of help with homework plus supervising play dates is more than I think should be expected of another parent or even a college age kid, unless you get very lucky and find a college kid who can really commit to five days a week of pickup, tutoring and supervising children. (Nothing against them -- our best sitter ever was a college student-- but their schedules do change during the year, they often cannot do anything five days a week at a set time, etc.)
I would look on nanny web sites but be aware that most nannies expect full-day work. You might find one who wants to shift to part-time and that would work for both you and her. You will need to check her driving license status, her driving record (does she have tickets? Especially for moving violations?), her criminal record if any, and a ton of references. Do you provide the car? Pay for gas in her car instead? Etc. Be sure to have a written contract that lets you fire her at any time "at will" with notice, and a trial period at the start so you can both see how it works out.
The problem may be with homework help -- you may find many nannies do not have experience doing this, and are not going to be comfortable being held responsible for helping with homework. If he does not get things turned in, or his grades slip, will the nanny or babysitter be held responsible by you? How would that work? Any candidate would want to know.
They might be willing to ensure your son sits down at a certain time with his homework but you might find they are not really able or willing to get very involved in helping him with details. Bear that in mind.
Being involved in his homework keeps you in touch with what he's doing at school. Anything you can do to ensure you are the homework helper would be good for you both; you will be much more "plugged in" to what is going on during his day if you can be the homework helper. Maybe a nanny can ensure it gets done but you can go over it with him when you get home.