We live with food allergies here too. My son is contact allergic to milk, so if he touches a table where milk was spilled and not cleaned properly he will have a reaction. So I do see your husband's point of view about the daycare. Even if it is a peanut free zone, you do have to be concerned what the other kids have on their hands, etc.
That being said, do you know, and trust another mom you could trade a few hours of babysitting with? Maybe at your house so you are not worrying if your son is in their home and they have peanut items in the house.
All caregivers need to be trained in the use of an Epi-pen and what your "action plan" is concerning any reaction he may have.
They also have tons of stickers, alert bracelets, onesies, t-shirts, etc that say things like "please ask my mommy before feeding me, I have allergies"
When my child was first diagnosed I had a button I could put on his shirt, even to remind relatives NOT to give him anything.
You could put a post on Craigslist, seeking another mom with a child with a peanut allergy and see if she is in the same boat as you. I suggest this because I saw a mom that did that. Or see if your allergist has any type of bulletin board in their office and see if you can put up a sign there.Then trade sitting with her, believe me, peace of mind your child is with someone that will immediately know how to respond to a reaction and know to check every label, every time (ingredients can and do change) will make you feel so much better.
My own husband would not feed our son unless I left the food ready until he really got the hang of the whole allergy thing!
If you cannot work due to the child care at the gym, and have NO other option, then your husband needs to recognize HE needs to be home with the kids and figure a way for you to carve some "YOU" time into the week.
Then it gives you that chance to get out and do some errands, or exercise, etc.