I left my job to be a SAHM. I'll admit in the beginning it was a difficult transition for me. I missed working, the people, and having a set schedule and structure, sure things popped up, but my job was always fairly predictable, that's definitely NOT how motherhood is. After a few months though, I started to get the hang of it. I absolutely love being a SAHM now, and hope I don't have to go back until our kids are in school.
I highly recommend getting out of the house every day, just to do something. Whether it's for a walk, to the grocery store, playdate, whatever... just get out. It will help stimulate your kids and help you keep your sanity. I also second getting time to yourself, it's really important.
Also you said, "Anyone else leave their careers behind to be better moms and a better wife?" Don't think that you're a bad mom or wife if staying at home doesn't work for you. It's not for everyone. So, if it doesn't work, it just doesn't work. Simple as that. Don't feel bad or get down on yourself if you want to go back to work. Working mothers are fantastic and loving mothers too.