Have you ever been unfaithful to your husband? Has he been unfaithful to you? If not, then you have EVERY right to go and spend some time with the girls. Maybe start out with a class at the local workout club, lunch, a spa day, manis and pedis, something he won't find "threatening." Then, you can expand it into happy hour or dinner (but not both to start with), and eventually he will get used to you taking time to recharge your own batteries, and he'll be OK with you going out with girl friends.
I find that men, at first, have a hard time accepting the fact that we actually want or need to spend time on ourselves. Our job is to take care of everyone else. And, we do that very well 24/7. However, if we don't stay true to ourselves and our needs and wants, we will become resentful, moody, etc. Little excursions like this help us reconnect with our friends, family, and most importantly, ourselves.
When I go out, I make sure to go early, have dinner ready for the family, call to say goodnight to the kids, and come home early, so he can get a little (or alot of) time with me, too. I've found that even just going for happy hour, and getting home in time to prepare dinner, does amazing things for my attitude and outlook on life and being a caregiver.
My advice, ease into it, and he'll be OK with it.