From what you've said, you have a very structured morning. After naptime, do you have the same kind of structure, or is it kind of a "play it by ear" type of deal? He may just need a schedule, so to speak.
Does he ever play by himself in the morning? You commented that you have a lot that you do together in the morning; maybe integrate a ten minute time for him to play before morning nap where you go to the bathroom, make the beds, wash the dishes, check your email, etc out of his sight. That way he'll know that the daily routine is still in play, you're just not there for a moment. That might make the afternoon transition a bit easier.
I personally think it's just a transition phase; new things can be scary at first. If he keeps whining, take a few minutes off to play and say that you need to get some housework done, and as soon as it's finished, you can play again. He may not be able to understand the words, but he can understand the tone.
I know how tough it is when you're frustrated, but it's just the age, and with diligence, it will work out. He may just be missing his "Mommy Time". :) Help him start a new routine for the afternoons, maybe even give him something special like a favorite song to listen to, a video to watch, or a special toy that only comes out in the afternoon. And if he needs to sit and scream for a while, let him. And if you need to give yourself a "time-out" somewhere, go ahead. Call a friend, take a long potty break, read your book, whatever. No one said mom's weren't allowed to vent once in a while. :)