Well, let me preface by saying that I do not have a good relationship with my daughters dad either. For a while we used a mediator as we couldn't even have a conversation about pick-up without a full-fledged fight - after 5 years we are now able to do most communication via text (you only get 160 characters at a time!!!!). We also have an inter-state custody parenting arrangement.
I believe that you DO have to take your ex's opinion into consideration ONLY because if the arrangement isn't work-able for him he won't take the time with your son and then they won't build a relationship. You'll be mad cuz he never takes his son and he'll be mad cuz you didn't make it easy and you are right back to where you started. Although, BELIEVE ME, I understand your position.
If your son won't fly, it's going to be more difficult. It's about 25 hours on Amtrack from Miami to Philly so the only thing I know is to alternate taking him on the train. However, he's 8 so that makes him a bit more logical and you can maybe show him some statistics on safety. My daughter has flown as an unaccompanied minor for 5 years. It's a 2 1/2 hour direct flight and she is 100% safe (I also tend toward overprotective. She is 10 and I don't let her out of my sight). But I take her to one gate, she watches a movie on her "special plane only dvd player and brand new movie she picked out", eats a snack and then voila! her dad meets her at the other gate. She is in the front seat of coach class where everyone can see her all the time, so you don't have to worry about predators and stuff.
Our visitation schedule is as follows:
2 weeks at Christmas in odd years
5 days at Thanksgiving in even years (this is based on school sched)
1 week at spring break every year
1 week over father's day (can be combined with summer visitation to extend summer visitation to 5 weeks)
4 weeks over summer school break
- summer visitation to include her birthday in even years (she is Aug b'day)
One additional 4 day weekend based on school calendar.
We split the ticket 50/50 including any bag fees, UM fees etc. I can give you more details on the logistics of payment etc if you want to message me - don't want to bore all the mama's with extra details :-)
I can tell you that even with this VERY liberal schedule..... he has only taken her every spring break and until this year (when he took her for all 4 weeks cuz he just got married last fall and wants to show he's a good dad) he only took her one week each summer. That's it. So, my suggestion is to be liberal. If he hasn't been an upstanding guy he isn't going to change his spots - but YOU will have done everything you can to allow him time with his son.
The other thing that you can do is have his dad come to Miami and spend 5 days (or a bunch of weekends) with your son. If he builds a relationship maybe your son would get over his fear of flying if dad is on the other side waiting. 1/2 his hotel will be comparable to 1/2 a plane ticket anyway.
Good luck!