We have done a ton of flying with our kids and I am going to tell you right now to drop the idea of bringing the car seat on board. IT IS NOT WORTH IT! If there happens to be another seat onboard available you can strap the older one in. Just check both car seats in. The stroller won't be a big deal, you just bring the kids up to the gate, fold up the stroller, roll it to the door of the plane and the steward/ess will take it and store it for you.
As far as carry ons, bring a small play bag for each kid, two big ones are hard to manage. Put snacks and SUPER fun things in it, plus any necessities. One thing I learned is to get new things and toys and games that are out of the ordinary for interest purposes. As far as carry-ons for you guys, keep it as small as possible. Believe me it is much more difficult to manage kids and stuff on a plane than you think and other people get real testy about waiting for you to get it together. I have actually been yelled at before and it just added to the stress of trying to manage everything.
As far as the other baggage, if you can keep it to two large bags and two car seats that you check in you won't have to pay anything (I think you have to keep the bags under 50 pounds which isn't too hard).
In short the hardest thing about planes and kids is getting them around with stuff, so the smallest amount you have with you the better. The plane ride itself really isn't that bad.
I know you are not really up for drugging your kids, but I have found that their little ears get plugged up and it hurts, so I typically give them a decongestant or Benedryl before we board to help equalize their ears (I checked with the doc first and it was A-ok). As we all know, Benedryl makes them sleepy so maybe you can get the tired babies along with it. = )
Good luck and enjoy your trip.