S.,
Have you retained an attorney? If not, you really need to - an attorney can handle all of your ex's [or his family's] comments, threats, etc., and will give you much more confidence in the situation.
If you can't afford one, locate an organization or group that can assist you with either providing legal aid, or something like that. You shouldn't have to go through this alone!
I had a "fun" time going through the whole custody/child support thing with my ex, too. He denied that our daughter was his, even though he knew very well she was. He sees her off and on, but does pay his child support, mainly because he didn't want his wages garnished, and his employer to know!
Do see if you can get some sort of representation - someone who knows how to play the game, and someone who can tell you how ridiculous your ex's/his family's threats are. They're threatening you now, hoping you'll give in. DON'T! If you feel you need to protect your child and your custody, do whatever it takes.
If your ex wants to be a positive person in your child's life, that's one thing, and it doesn't have to mean that he has to be a part of yours - I'm supportive of my ex's time with our daughter as long as it remains a positive thing for her. But I'm also lucky he lives in another state and just doesn't have time to see her that often. But before the case was settled, and he was saying all sorts of crazy things, I told him he could just talk to my attorney - we didn't need to discuss anything.
GOOD LUCK! Stay strong! You're doing the right thing, and I hope you can find an attorney who'll stand beside you and protect your rights, AND the rights of your child. Don't let those threats get to you - that's what people do when they feel weak. They threaten!
Take care, and best of luck to you tomorrow. Any mama who's doing the hard work of raising a child should shine through in court, and usually, the court sees that. The ex and parents can try to bully you, but that doesn't fly in a court of law.
Best wishes,
J.