Coming from London and just visited this summer with my then almost 3 yr old, 6 yr and 12 yr old. I can tell you London is huge, and the traffic is really bad. There are lots lots different things to do, but you don't want to be spending all your time traveling on the buses or the Tube.
I would definitely take a trip on the buses and the tube is the best way to travel around but avoid rush hour, ( 7-9.30am and 4.30-7pm). In fact the cheap day travel cards start after that time in the morning. My boy was scared by the amount of people rushing about on the Tube.
Make sure you bring your stroller there is a lot of walking to do in London, sometimes it is easier when things are so close together.
London Zoo is a great place to visit, Hyde Park, Regents Park and Hampstead Heath are good and big parks to Visit. A visit to Trafalgar Square so he can chase all the pidgeons! We did an open top tour from Trafalgar Square, the good thing is you can get on and off as you like, which is a good thing as he won't tolerate being on the bus for the 2 hours it takes to go around, ours didn't! We did the London Eye which we enjoyed, but I'm not sure how much he remembers. Madame Tussaud's is another that can be done.
All the museums are free. The Science Museum is probably the best at this age, my kids had a blast there. The only thing you may have to contend with is school trips! Convent Garden is also a fab place to visit and the have street entertainers there, he would love that and there are shops there for you to indulge yourself!
Can't think of anything else at the moment but if you want to email me with any questions, please feel free to do so. Bring your camera and take loads of pics. Enjoy your trip, writing this is making me feel homesick!