hmmmmm. There's a lot going on here. First, I think it's in your best interest to put something official in writing since you all can't agree and your son is suffering by not seeing his dad. It doesn't matter who is to "blame", it is what it is.
First.... I believe that in an ideal world EVERYONE would do what is necessary to make sure kids have as much time as possible with each parent. However, we don't live in an ideal world.
Essentially..... yes, you moved and agreed to bear the brunt of expenses to get your son to see his dad.... therefore.... Your responsibility. You made an agreement and you have now backed out of it, because it didn't work out they way you thought it would. Yes, it's tough for you..... but, again, your son is the one who really suffers.
You REALLY need a child support order and a parenting agreement and this all to be spelled out and filed in court. This actually protects ALLL THREE OF YOU. Because you are trying to justify you going back on your word by making his dad out to be the bad guy. He's probably saying you moved away and now aren't holding up your end of the bargain so he can't see his son. If this was spelled out each of you would HAVE to do whatever it took to get your son time with each parent or you would be in contempt of court.
Here is what I can tell you....... I live 2 states away from my daughter's dad (16 hours by car / 2 1/2 hour flight). We have a visitation agreement that stipulates he can have the following:
*Every Spring Break
*4 weeks in the summer (include child's b'day in even years)
*Thanksgiving break in even years
*Christmas Break in odd years
*Father's day extended weekend (4 days) unless it interferes with school
*One additional 4 day weekend that does not interfere with school or mother's holidays.
We split travel costs 50/50.
One additional thing.... I urge you to change your mindset. It isn't your responsibility to bring him to see his dad every other weekend because you moved..... It's your responsibility to ensure that your son is able to have a relationship with his dad BECAUSE IT'S WHAT'S BEST FOR YOUR SON. Doesn't matter if it SUCKS for you.... or SUCKS for your ex. You do it for you son.