I’m a big (bigger?) believer in the following, whether for personal life or work:

Set goals.
Set guidelines to attain that goal.
Regularly reevaluate the goal.
Constantly evaluate the guidelines.
Adapt according to circumstances.

And remember; winning every battle is not as important as winning the war.

If I apply this to your situation:

Your goal is Plan B to get closure. Either to end the affair or end the marriage.

Your guidelines are the ones you set forth in writing to H and the ones you apply to yourself.

Every now and then you have to reevaluate the goal. Not constantly – have to remain focused. Too many people end up going for something they no longer want or even know what is.

Constantly revaluate the guidelines. If they work use them. If not find new guidelines.

Adapt. Like now you have the guideline not wanting H in the house. He still has some stuff there so he has a legit reason to come by. Your adaptation would be to remove the legit reason OR sway the guidelines in a minimum way. The main guideline is no contact with you. Him not sleeping in your house is secondary to that. Him not entering the house is third to that.

Your best reaction would have been to remove the legit reason i.e. get rid of his stuff. Then remove the possibility of him coming in i.e. change locks. Other than that it is just a case of minimizing the damage. You did that in a way by not staying – you lost by letting him enter, letting him stay. Those two are really small losses compared to the mental loss of you wondering what he did, why he pushed things around etc. There you are letting him control your mind!

But don’t worry. This was just a small skirmish in a big war. He might just possibly have won this skirmish but you are still fit for fighting. Look at the positives; he has now removed the last legit reason he might have to enter the house. You can change locks with a perfectly clear conscience.