My advice would be for you to not let your son quit. And don't quit on him either! It's important to know that by letting him quit, you are only enforcing his behavior. As the parent, you should be the one controlling the reins; PUT YOUR FOOT DOWN! :-) By letting your son get away with his behaviors, you are telling him that he can quit anything or get his way with things as long as he cries or acts up. Also, I think it is very important to explain to your child that they HAVE to at least TRY something AND follow through with it; especially when it comes to sports. If they don't like it after they try it with good effort for the WHOLE season, then they don't have to do it again. But stick to your guns.
This was my parents phylosophy, and coincidently, it was my wife's upbringing as well. We follow these same principles for our daughters, and it has worked miracles.
My wife and I have twin 4yo daughters who have been in sports and activities since they where about 1 1/2 years old. We started them in gymnastics simply for the purpose of socialization. At that age, all of the classes were parent/child. However, after a year or so, they moved up to the child-only classes. Our daughters HATED not having us in the room and they definitely threw some fits. For a long time, we would have to sit in the room with them while all of the other kids played on their own. They would participate a little, but, always had an eye on us...just to make sure we didn't leave. But, we were very persistant in enforcing and encouraging them to participate with the class.
My wife and I would talk with them daily about being "big girls," in class and how proud we were of them for trying each week...but also how awesome it would be to see them go in to the class all by them selves. At times it did feel like a giant waste of money, but, each week they showed just a little, tiny bit more improvement. My wife started to slowly move closer and closer to the door each week. Then, one class she slipped out. Once our daughters realized she was gone they freaked out. But once they saw that she didn't "leave" them and that they did it all on their own, they were estatic to be "big girls."
We made a HUGE deal about it, commended them with great enthousiasm, called their grandparents, aunts and uncles, and let them tell everyone how "big" they were. They grew leaps and bounds following that day, and are now little social butterflies no matter where we go...school, the doctors, grocery store..everywhere!
We never let them quit, and they grew to LOVE gymnastics, as well as dance, soccer, and swim club. I am SO glad that we stuck to our guns with gymnastics, and was worth all the stress and money to see them blossom that day.
Stick with your guns, praise him for his accomplishments, and don't be afraid to be embarrased when he has his moments! It's very easy to get lost in our days and forget to tell our kids "great job" when they are good. Because, it's VERY easy to let them know when they are being "bad."
Best advice my wife ever gave me..."Just remember...they are only 4 years old." Always reminds and puts into perspective how little they know in this world...and how much WE need to teach them. Both with our words, and with our actions!
Best of luck to you, hope this helps even if just a little :-)