I'm sure part of the problem is you caught him. He didn't confess. How much longer would this have gone on if you hadn't caught him? How much further would it have gone? Is he sorry he did it, or sorry he got caught? How do you trust him when he still gets mad like this is your fault when you don't trust him or ask questions? He's defensive and angry, that doesn't help build trust.
I've been there too. My experience has been when I DID snoop it was because I knew it was happening and he was lying to me... being right didn't make me feel any better. It did help me feel confident I was making the right choice when it ended.
Ask yourself.. if you did spy and found the worst - what would you do? Would you really leave, or would it just be that much more you were trying to forgive? Would it help, or make things worse? If you found proof he was cheating are you ready to tell him to leave?
Emotional cheating is cheating, sex or not - in my opinion. I don't believe "once a cheater, always a cheater" -- if this is his only misstep, and he's committed to working on your relationship... understanding trust takes a long time to rebuild so he needs to be patient with you. Continue your counseling and you guys can work through it. Assuming YOU want to. My opinion is MUCH different if this is the third, fourth, tenth time... I do think some people are chronic cheaters and they won't likely change.
I guess the real question is what do you want? It's tempting to want things to be like they were before you found out, and to keep wishing he didn't betray your trust. That's not fair to either of you - because you will hold on to a lot of anger, and you won't move from it. Decide what you really want here. If you want to work on it, then do. If you can already see yourself ending it, then maybe thats where you need to be.
I'm sorry you're going through this. I can honestly say I have been on both sides of this. Not something I can brag about. I cheated, once on a previous relationship - I was honest with my partner and told them. I never have again, never would again. I have dated a guy that was a chronic cheater and liar, made me feel to blame for catching him doing things. If I were in your shoes I would schedule a counseling session alone with you counselor and talk it out with them. Talk about how you're feeling and see if you can work through it.