Hi L.. I've had this issue with all 3 of my older kids, who are now 6, 4, and 2. I've found that the best thing to do is be as honest as possible when you're leaving the house AND to be ready to make a quick exit. Have the sitter completely prepped as to all the instructions by phone so that you don't have to sit and talk with her once she arrives. Have all your things ready to go, and go through a list of things you might have to go back into the house for (phone, purse, keys). When she arrives, tell your son you love him more than anything in the world and hope he has a very happy day and that (MOST IMPORTANT) you will see him very soon, right after dinner/nap/whatever he can associate timing with. Then exit QUICKLY!
Most importantly, do not coddle him through this. Do not give him reason to believe it really is an awful experience by snuggling and going back to him when he's crying. Give him your happiest face and voice, and GET OUT! In my experience, the crying might last 10 minutes the first day or two, but after that it literally lasts seconds. I've heard my 2-year-old stop crying as I shut the front door. My understanding is that's mostly a show about making sure you know that he's going to miss you so that he can be assured you'll come back.
Do not feel guilty about leaving him when he is crying. You know that he is with someone safe and that he is not in any danger. The only way you can make sure he knows this, too, is for him to feel your security by having you leave him with happiness and no stress.
And also it's important for you to remember that this is a phase and that someday he will actually look forward to his sitter coming to play. My older three get very excited when I tell them that their sitter is coming because she can actually play with them instead of playing while folding laundry, cooking, doing dishes, cleaning the house, etc!
Good luck to you!