Putting aside the whole situation with your ex, I'd base my opinion on the fact that this is not a "road trip" (in the usual sense, where people drive somewhere in order to see things along the way). This is a long drive to an event, with just one driver and three small children.
And then, when they arrive, the children must attend a wedding. They'll have to be neat and clean, well-behaved, and attend a ceremony of someone they probably don't know. If the ceremony is going to be a traditional one, in a church or venue where the children will be dressed neatly and be required to sit quietly, that would be very demanding on them, especially after such a long trip (if it's a casual outdoor wedding where there will be lots of other kids, then that might be less stressful for the kids). Have your children attended a wedding before? Are they capable of sitting quietly in church (or a wedding venue), and familiar with wedding etiquette?
All of this requires your ex to get to the wedding on time, ensure that the kids are appropriately rested, and dressed for a wedding, meet lots of relatives, and then do the whole return trip. It doesn't sound enjoyable for anyone.
If they were taking a leisurely trip where they could stop and look at attractions, and then enjoying a family reunion picnic, that might be different.
Basically, it isn't a road trip to a family reunion. It's lengthy travel to a wedding of someone whom the children do not know, and lengthy travel back. I wouldn't subject such young children to that, but I wouldn't say no to an actual road trip in the future, where your ex can say "hey kids, look, up ahead is the world's largest gum ball! Let's stop and run around and have fun!". Or "we've traveled far enough for today. Since we have no agenda, let's stop here. We'll find a motel with a pool."
I'd be sure to find out what kind of wedding it's going to be (who will supervise your children while your ex is visiting with all his family, what would your children have to wear, is it the kind of wedding that takes place in a church, with a reception afterwards, etc).
And if it's traditional, I'd make it clear to your ex that you're not objecting to his trip, simply objecting to making your children endure such a long ride and such an event.