Shelley,
That's a tough situation to be in. My opinion is that it is obvious that your husband is out of your relationship, although sounds like he doesn't want to say it's completely over. If like you are saying his affair is JUST emotional, which sounds like it's nothing then he should have no problem terminating it. However, personally I think that emotional affairs are even more dangerous than sex affairs, like what does that mean "emotional", is he in love with someone else? And in that case I would find it hard to belive that it is not also physical. Does your husband stay physically close with you? From what you said it sounds like your husband doesn't want to be close with you, physically or emotionally, I mean he moved out, right? The only reason why he is probably still saying that working on the relationship is still an option for him is because of the baby, I think.
All I want to say is that don't give him any leniency, he doesn't get to have his cake and eat it too. You are absolutely right, he doesn't get to have two parallel lives.
Even if you ask him to end contact with that third partly, I find it really really hard to believe that he would do that completely, I mean would you yourself be able to just cut contact completely with somebody you are in love with, or emotionally attached to (like your husband)? Your husband didn't have a problem moving out, so I guess he is fine with not having contact with you; is he still seeing that thirt party? Don't be a push-over, don't just let him have his way, I know it might mean that you will be alone, but you do deserve some respect, and actually you deserve a lot of respect.
I would suggest for you to go ahead and start looking at the options for yourself and your son in case if you get a divorce, like your finances and child support for your boy, and visitation hours. Now that he moved out he basically left you in charge of your and your son's life. Also please consult a lawyer. Right not it might not seem like it's a big deal, but don't wait until he said he decided the you two are done, cause then his first priority would be to get his financial independence back and then HE would go to the lawyer first.
Be strong and good luck to you