M.,
Look at tummy time from your son's perspective. He's in a position where he can't really use his hands, he can't see anything but the floor next to him, and it's rather unfamiliar. When you look at it this way, it does kind of suck. I'd cry too.
This isn't a bad thing. If your son doesn't like tummy time, he'll fight harder to learn how to fix it. All that struggling to move and to puch his head out of the floor is good for him. It helps build strength for rolling and eventually crawling.
As far as reaching milestones on time, your doc's right. He's going to roll later than babies used to when they slept on their stomachs. But, he's probably going to roll right on time in comparison to modern babies who all also sleep on their backs. There's no medal for this, so does a couple of weeks really matter?
Take it from one who knows, my son is now two. Mobility is NOT the holy grail new moms think it is. It's a big pain in the neck. I would give my left arm to go back to immobile infanthood for just one afternoon! To just once find my son in the same spot I lift him when I went to the bathroom. To not have to wonder if he's sitting indian-style in the middle of the dining room table calling Nana on the cordless phone. To not walk back in from filling his sippy and find him dancing to the Dora theme on the back of the couch. Not escaping out the back door because we've figured out the lock on the screen door, or scaling the changing table to get the wipey box and throw them around the nursery like a snowstorm. This too chall be yours in a remarkably short period of time. If he waits a week to roll over, put your feet up and grin.