T.S.
We did a long weekend in Virginia Beach over spring break this year and we ALL (me, SO and DS-7) loved it. DS says it was his favorite vacation (and we've been to WDW twice in the last three years)!
It depends on what you guys like doing. I like to feel relaxed on vacation, so we stuck with Virginia Beach itself, but if your family LOVES theme parks, do it. There are lots of hotels right along the boardwalk... I'd definitely stay in one of those rather than somewhere in town... being able to walk out the door onto the beach is really a plus. I think we stayed at the Fairfield, which was great. We had a little suite so DS had his own space and breakfast is included every day (which certainly adds up as you vacation with more people).
-Rent a bike surrey and tour the beach front.
-Rent fishing poles and see what you can catch off of the pier.
-Fly kites on the beach when (if) it cools off in the evening.
-Hang out on the beach (that's why you're there)
-Go to the Aquarium (definitely do this... is a nice one)
-Take the boat cruise from the Aquarium
-Rent kayaks or paddle boards and paddle around.
-Eat seafood
DS is asking to go back, so it's probably in our not to distant future. I'd do Williamsburg with him but ONLY because its along the drive home for us... Depending on how long your planning to be in VB, I don't know if it's worth it to spend travel time getting to another city... but maybe it is for you guys... I've never been to Williamsburg but I've wanted to since I was about 9 (American Girl dolls had a whole thing there for the new Felicity doll around that time) but SO grew up here and has been a million times and hates it... go figure.
HTH
T.