D.W.
Where are you staying? That will determine a lot of what you do.
If you're staying downtown, it will be a little easier. If you're in the suburbs, you'll probably want to ask the concierge at the hotel (I worked for a Marriott in the VA suburbs for 4 years in college and doled out a lot of advice to visitors) for advice on the best way to get around. They may have a shuttle to take you to a Metro stop or may be able to recommend a less-expensive sedan service. Driving is not the best option if you're not familiar with the area, and parking can be rough.
We went home over Christmas to see my whole family who's still there. We took our 3 year old and then 20 month old to many of the Smithsonians. They weren't as interested as I thought they'd be.
But, I'd recommend the Natural History Museum, the American History Museum, both Air and Space Museums (the National Mall and Dulles). If the weather's nice, I'd recommend walking the mall and the monuments. If it's a trip relatively soon, the Cherry Blossoms are always amazing. And, they may enjoy the National Cathedral, but it may be better for older kids.
The White House is less kid-friendly and more secure.
If you do have access to a car, I'd highly recommend Arlington Cemetery and the Custis Lee Mansion, Mount Vernon (George Washington's home), Old Towne Alexandria, and Manassas Battlefield (Battle of Bull Run).
There are too many things to do there than are possible in a few days. Having grown-up there was amazing, and though my family's been there for 32 years, I've never seen most of the sites myself.