Having been in your shoes a couple of times, and if I were in them again, I would not take my husband back into my house until I knew for sure that things were over 100% for good with the ow. I read your history, and your husband seems to have the classic cakeman syndrome, wants his cake and eat it too, and he plays the "I don't know what I want game, you or her." From my experience, I would never give a man power and control over my life by letting him have the choice of me or her. When I was married, and I took my husband back, and I gave him the choice of "me or her", I still would feel like I was second best. He kept doing it over again. I would not hide any of what you are feeling, if you have a question for him, ask it no matter how uncomfortable and squirmy it makes him feel. If you take them back into your house without letting them suffer the consequences of their actions, then they don't ever change, they will keep on doing it again and again as long as they feel they can come home to you whenever they are done playing house with another woman.
I played the role of the other woman for a very short time. The man that I was involved with was not married, but living with someone. I was divorced when the affair started. A man that I worked with told me that he had feelings for me and that he wanted to date me. I knew that he was not married, but I didn't know that he was living with someone. I didn't find out until after we were intimate for the first time. I immediately tried to end things between us, but he told me the same old story, he wasn't happy at home, he didn't get what he needed etc, and that he was planning on leaving. It did not take me long to figure out that he was just playing a game with me and I ended things with him for good. We still worke together, and we are still friends, but that is it. There is nothing physical or anything between us anymore. He still tries to tell me the same old things, that he loves me, that he is not happy at home etc. It has been over a year since our ema ended, and I am now engaged to be married to a single and available man.
A lot of people who enter into ema's, do so because they lack the courage and self esteem to end one relationship before they start another. That is a fault in themselves, not their spouses. They will usually blame the spouse for their actions because if they really blamed the person at fault, themselves, then they might have to actually look in the mirror and really see themselves for what they really are. A lot of them don't want to face reality, so if they can place the blame on someone else, then they can keep on doing what they do. It is a circle that usually never ends unless the person at fault changes some things about themselves.
Good Luck