To be honest, if he screams, let him and ignore him. When you aren't worried about it and it isn't getting him what he wants he will get it. I used to always do the car carts at our store, if yours have it, my kids LOVED those. They are both big now so they have to walk.
I hate to say it but it shouldn't break your heart, he is a toddler now and it is very normal behavior. I quit worrying about what others thought and it lessened my stress and then my children calmed down. I mean I would remove my children if it was a movie, dinner out, but at the store, I just go about my business. When they got older and acted up, I literally did time outs at the store!!! I made it clear time outs can happen no matter where we are! Kids protest when they don't get their way, it will happen about a million more times and you have to just ignore him.
It isn't safe at his age to let him out of the cart anyway. He could pull something heavy on himself, trash a display and so forth. Buckle him in and do your shopping. It sounds hard core but if you have to shop you have to.
Try finding a store with the cool cars so he can "drive" through the store. One thing that worked when my daughter was young was letting her take her Little Tikes shopping cart along and her baby doll. She filled it with easy and safe products and went through the check out with me. I also go to the cookie aisle in the bakery section last, the reward was a cookie if they were good while shopping. May sound like bribery but I look at it as a reward for being patient. Our store gives one free cookie. So check into that. Make sure he goes when he is well rested and not hungry either as I know that makes behaviors esculate.
Even at 4 and 7, my kids hate the store, but some days I have to take them with me. For the most part I just try and do it when they are in school though.
Good luck, part of toddlerhood and he will get it, it is a phase and protesting is very normal. Hang in there!
I would if it is that awful, go without him, leave him with your husband and you go alone.
When he is a little older and starting counting, have him help, like put five apples in the bag. My kids love the twisty ties in the produce dept and I challenge them to make things while we are walking through the store.
When your new baby comes, he can still ride in the car carts and you can put the carrier on the top.