This is hard, and it's not going to get easier any time soon.
and I do not want him in the house again.
So this is the kind of thing you should be trying to avoid. Around here, it's called Love-Busting (or LB'ing). In his mind, you confirmed for him that you are an angry unreasonable person. His response (being angry and defensive) was pretty typical.
I know that this is really hard, and you're having to absorb a tremendous amount of stuff in a very short period of time. Huge amounts of raw emotion make it hard to think straight. Plus, a lot of what we tell you will probably seem counter-intuitive to what you are feeling.
The advice you're currently being given assumes that you want to save your marriage. Based on what I've learned and know about this place, I think your marriage is very salvagable. Have you thought about whether you want to save the marriage or not?
Changing the locks is a step towards divorce. It's probably just what the wayward spouse in him wants you to do.
Any time you're confronted with the alien, it will trigger a response from you. You'll have an adrenaline rush, wind up shaking, and you'll really really want to REACT. Do Something to make it better. (Dealing with these kinds of triggers is specifically what my Remember to Breathe line is about--it's a reminder for me as much as anything.)
Typically, the instinctive response to the trigger is out of emotion and not the one that is the most constructive toward rebuilding a marriage (e.g., giving him a piece of your mind, changing the locks). This is why you should avoid contact with him if you can until you have a better idea of what you're up against. Put a Plan in place. Probably Plan A.
Have you read about Plan A? Have you thought about calling the Harley's call-in radio show? Or setting up an appointment through the counseling center? There are a couple of books that can help--Surviving an Affair, and His Needs, Her Needs. Both will be useful.
I hope my tone is not off-putting. I made a lot of mistakes at the beginning because 1) I didn't find this place soon enough, and 2) I didn't listen to what I was told when I got here because I was in my own fog. I don't know that things could have been any different for me, but I still try to help people avoid the mistakes I made.