N.S.
There are certain people who only find you desirable when you are unattainable. This type of person is incapable of love, psychologically speaking. They find you attractive because they don't need to be accountable to you. It would be foolish put your current marriage on the line for a man who clearly couldn't commit to you in the first place.
Don't you see there is something already wrong with this situation in that that he's disrespecting your marriage to another man, toying with your emotions, AND is obviously romantically involved with another woman? This guy has boundary and commitment issues.
Don't let those pregnancy hormones and your imagination ruin your life, and the future of your children. You are clearly "projecting" your own feelings onto this man. You are probably in love with something that doesn't exist. Often people who can't commit "get off" on controling others through emotions and romantic entanglements. He'll say and do whatever you want until you are ruined. And once the sport and fun of it all is over, he'll be gone. This is about control not love. I guarantee it.
From here on out, short of court appointed meetings, stop seeing this man. You have no reason for non-business related meetings. And when you do, make sure hubby and kids are there. That'll keep the buzzard in check. Otherwise, he'll just string you along, as long as your willing to be his fool. Hopefully your current husband isn't hip to what's going on. Most people have a sixth sense about stuff like this. Don't let things go too far. You will be the one to lose if you invest any further energy into the ex, and send bad vibes to the man who is actually there for you and your kids.
Afterall, the ex probably won't marry you again if you do leave your husband. And if he does marry you, he probably won't be faithful.. he clearly doesn't respect the marriage bond, no matter who it is.
Good luck