Your husband is behaving like an absolute idiot and jeopardizing his business. He has the equivalent of a schoolyard crush and this woman whom he's only know for TWO MONTHS could absolutely rake them over the coals for sexual harassment, etc. etc. As the wife of a lawyer, my brain exploded when I read that: "he has written her an apology letter promising to tell me, his wife, and to move on." So, basically he is on the record for having harassed her. I'm not saying that he should deny it if it happened if she claims harassment, but things in writing can be misconstrued for high $$ value, if you know what I mean.
Our marriages are cyclical, the fact that he is attracted to another woman isn't shocking and I wouldn't care so much that he has a crush. A lifetime is a long time to stay together that's why we essentially enter into a contract "forsaking all others"...not because it changes your normal instincts, but because you agree on a mutual intention to do so. It's not that you're being overly sensitive. You deserve to be upset and I don't blame you for crying but, you know, he's acting out during a time of stress, it's NOT about you.
And I realize that's not particularly helpful since that's precisely the problem--you don't seem to factor in...but he's behaving in a very self-centered way and I'd treat him like a child who is being self-destructive. He, unfortunately, needs to be parented right now...not wifed.
Now, here's my suggestion. Go to counseling. I don't think your marriage is in some horrible crisis, but this is too touchy a topic to deal with between the two of you. You are emotional, he is acting out, you need a third neutral party..mostly for him to talk about whatever he's going through since he can't manage it himself and he's being a dumb jerk.
Good luck, mama. I hope things work out ok. If your husband's business has an attorney I'd have your husband call him up and tell him what happened in order to get ahead of this. Anything can happen, you need to be prepared and HE needs to understand how incredibly dangerous his behavior is to his own livelihood.
Edit: read the response saying to fire the woman. DON'T FIRE THE WOMAN, especially until you get advice from an attorney. Oh my GOD don't do that. Quick story just to give you an idea of how sticky this can be: we know someone who hired a woman who turned out to be a serial victim. She would get hired as as assistant for these high powered executives, seduce them (usually on "business trips") and then essentially the next day claim all kinds of mental and emotional distress, sexual harassment...and demand a settlement or would sue. Generally speaking, something like this would require a small monetary "severance" payment and a "non disclosure agreement".