Hi Jae,
It's been years now, but some of the best games my kids played when they were little were made by Humongous Entertainment. Totally kid appropriate, educational, and really fun. You can download at least some of them to your computer for free, or purchase full copies. As a starting point, I found one of my kids' old favorites as a free download for you "Fatty Bear's Birthday Surprise." http://www.games2download.com/free-kids-games/fatty-bears...
Humongous made a series of the Fatty Bear games, which teach problem solving and preschool/early elementary school skills. You should also check out their Putt-Putt and Freddi Fish series.
Also good games to look out for are the Reader Rabbit games, Treasure Mountain, the Grade Adventure series (Kindergarten Adventures, 1st Grade Adventures, etc.). These teach a variety of math, science and reading skills at age/grade appropriate levels. As your son gets older, see if you can find games like Operation Neptune (math up to basic pre-algebra), Where in the World is Carmen Sandiego (geography), Where in Time is Carmen Sandiego (ancient through modern history), The Incredible Machine (logic and problem solving, plus very, very elementary physics), the Dr. Brain series (science and problem solving), and the Zoombinis series (logic and problem solving). These are all E rated games for the computer and are really, really fun, even though they may be older technology now. (Even I enjoyed playing them with the kids!)