We have taken the Toys Without Batteries path and I love it. I've been teaching kids for a long time and know A. how these toys do less to help our kids imaginations and play and B. my personal threshhold for the noise. This sort of play is less relaxing and is very restrictive in how they are used.
I cull through books and just donate what we don't want/need to Goodwill. Because of my work, I have a separate stash of Teacher Books my son can access when he likes, but they are kept apart from his books. I'm willing to have plenty of quality books, not junk.
The electronic toys: I'd say, keep the two he enjoys most, then do what suits you with the rest of them. Say, craigslist. Then, buy the toys you'd like to have. I'm a huge fan of blocks, and a simple block set (make sure they aren't too small) is great. You don't say how old your son is, but PLAN toys makes some nice wooden food sets, as does Melissa and Doug brand. My sister also says her older boys (4 and up) love the Kapla blocks, and you can get those at Target, I believe. If you are into construction, you can even save the cereal boxes and other things you might put in the recycling. Those are great for stacking, as well as cardboard tubes (two toilet paper tubes, tape, string-- instant 'binoculars'!) The PLAN toys Cone puzzle is also one of toys kids at my preschool play with daily. (We use mostly natural toys... nuts and rocks are included in our play, as are marbles....) A marble run (it doesn't need to be fancy, or wooden) is also great at teaching sequence and planning for building.
Oh, and I also rotate toys out of my son's room to another part of the house. He has been given a lot of good stuff, and only so much fits in. I notice what he's using most of the time, and put the other stuff away temporarily. Then, when he's needing some novelty, it comes back out and something else takes a break for a while.
I hope this gives you some ideas and PM me if you want more resources.