L.A.
Has he always been difficult or is this new? I know that 4 yrs is the developemental time for kids to use threats and ultimatums. I found the best thing to do was to listen for the message behind the words and then rephrase. (better for the kid to rephrase, but if he's resisting, hearing you DOES sink in.)
Maybe at 4 he needs more "control". I put that in quotes because he needs a lot of guidance, but perhaps he *thinks* he's ready for more autonomy. Be sure your son knows the reason behind what you're enforcing. Give him choices within your boundary.
Another thing that really helps many families is that you can give your rules and limits in a kind voice instead of the standard "threat tone". It doesn't take away anything. You are still enforcing or preventing. What it adds is that your words sound helpful. Human nature responds to "you must" with "I won't". Learning how to avoid this keeps that push-back trigger from kicking in. -- to add to the tone, you can even touch him in a way to remind him that regardless of this incident, you still love him. I've done this for years and it does NOT send the message that what he's doing is okay. It DOES help to re-ground him and calms them down -- which is where you'd rather have him be anyway.
For what it's worth, I wouldn't threaten with throwing away bikes and such. I'd just say something like "I can't keep you safe when you ride away on your bike. I'm going to put the bike away for awhile and we'll try again later. I love you very much and want you to stay safe. The bike is putting you in danger when you ride it away from home,so we're going to get that out of here." -- you still are removing the problem but now instead of the "you can't have" vs. "I want" it's now you watching out for him, which is what you're doing in the first place anyway.
Faber has several really great books that are easy to read and packed full of practical tips. "How to Talk so Kids will Listen" really helps lots of parents put the same rules into language that is more receptive. Another good author that is funny is Anthony E. Wolf. (You can find them at the library.)
Since threats aren't his enforcement and you'd probably have to escalate the severity of them over and over for it to have any effect on him, I suggest not using that tool and trying something else. Both of those authors will give you lots of new ideas.
Best of luck and congratulations on the new arrival!
update: When you talk to him, focus more on WHAT HE NEEDS TO DO and WHY and much less on why he's getting punished. In other words, focus on what you want to happen. Focusing on the punishment part brings attention to what you DON'T want to be happening.
also, if this has been a long term thing, a great book for helping you understand, teach and yes even appreciate his temperament is "Raising Your Spirited Child" by Kurcinka. The stubborn ones are a special challenge and this book (or audiobook) is fantastic. Determination will help him achieve his goals in adulthood. In childhood it can very well get him thrown out of a window. LOL