Hi L.,
The choice whether or not to baptize your child is yours alone. Nobody on this board should be counselling in matters of faith - that should be left to the people you are in community with (friends, family, other church members) - people who KNOW you and walk out your life with you.
Here is what I want to try to offer advice on: I have been divorced for 14 years and remarried for 8 years. It takes a long time to cut those emotional ties with your ex - especially if you have a child together. I have just recently learned in the past couple years (I'm slow) that trying to involve my daughter's father in matters that are my choices, is called "negative intimacy". You, in no way, shape or form want an intimate relationship with this person but you want, as a mother, for him to be involved in your child's life so you bring him in for an appearance of involvement.
In this case, you are not legally required to involve him in this decision. Why are you even considering his wishes? The two of you are no longer parenting together except for in very few, specific situations that should be outlined in your custody agreement. I doubt this is one of them so he, figuratively, should not exist.
It's AWESOME that you want him involved but lots of women make the mistake of enabling their ex to stay in their lives for the "sake of the children". I'm not accusing you of this but would like to just ask you to look at it (and maybe others areas) to see if you are including him in other matters that you shouldn't.
I longed for and tried to control this picture of an amicable divorce where we could all go to soccer games and have dinner together. I asked his opinion on things like discipline etc. that I shouldn't have. It can work for a little while (your child is little) but it is not sustainable - remember, you divorced because you couldn't work together - and it ends up hurting the child so much more in the end.
My daughter is 15 and she would have been better served if we would have just had a respectful, friendly but business-like relationship rather than tried to "co-parent". God bless those you can do it but divorce sucks and it is hard on the kids - period. Each parent has to own their involvement and do their individual best - not try to control what one another does in the other home (VERY COMMON MISTAKE!!!).
Over time, you will have to make sure that it is not disrespectful to your new husband to have this other guy making choices like this too. He signed on to be a parent to your daughter - not a spectator of you and your ex. You need to make sure you are honoring him in that.
OK, big topic -- good luck! You sound like you have a good heart. I just wanted to offer a piece for thought so that it doesn't get trampled on over time.