Oh my, I'm surprised with some of these responses. I have taught lessons for more than 10 years from infants all the way to adults. Four years old is not too young in my book. ;)
Some suggestions that I have (which may or may not work):
1. If you don't have any already, buy him some "Goggles". Target has some inexpensive Speedo googles for youth that tend to work much better than the cheaper kinds.
2. Take him to a pool with some friends or family and just play. Don't worry about putting his face in the water, because that will come. Play, play, and play. Show him that swimming can be a lot of fun. Do this a lot during the summer. Even taking him with kids that know how to swim so he can see how much fun they are having, etc.
3. Bring toys, and even little ping pong balls that he can blow around on the top of the water. Eventually you can work on blowing bubbles in the water.
Don't ever compare your child with another in swimming. But just to show you that a 4 year old is not too young, my little girl can swim a length of the pool. She doesn't have the breathing to the side yet, but all that will come.
Good luck!
C.