Hi K.,
I've been in the courts myself with my Daughter, you would not believe what I went through with her Father and they still grant him visitation. I will be straight up with you... he will get visitation regardless of pretty much anything. If you fight the decision it will most likely not go well. However as being there is no Father/Child bond, it probably won't go to unsupervised right away. The courts tend to like the releationship to be more established first. Don't worry about custody change he won't get that unless he can prove beyond a shadow of a doubt that your an unfit parent, which I doubt he can prove. You sound like a very concerned loving Mother who wants whats best for your child:) Do you live in Indiana? If so then it would be a good idea to look up the Indiana Parenting Time Guidelines. Heres the web site
http://www.in.gov/judiciary/rules/parenting/index.html
It tells basically all that you need to know regarding the parent times. Should he want more time (Which he can request)in order to get more time than whats stated in the Guidelines then you would have to agree first, if you do not agree then he will have to take it up with the courts. If at any time things get ugly and just don't seem to be working I would suggest getting an attorney, I know they can be very pricey but they can also be alot of help when it comes to these situations. Just make sure you ask around and do your homework on the attorneys first, don't just jump in and pick the first one you see, trust me on this... do your homework first. I'm sorry this is so long, I hope this has helped at least a little. If I sound pushy or rude please forgive me as that was not my intention, I just don't want to see you go through some of the nightmare my husband and I have been through with our children. I have custody of my Daughter, and his ex has custody of theirs, we've had problems from both of the other parents. Remember to stay as calm as possible in court, I know hard to do. I wish you the best of luck. Sorry so long.