I understand your distrust of your H. I am not going to give you an opinion because it is meaningless. I have endured 9 affairs in the last 8 years of our marriage by my count. There probably have been more because I only know about those because I was snooping to get the details to confront her with them.
Am I going to divorce her? No, because whatever I show her Jesus is going to show me when I see Him on judgement day. Besides I want to show her the love He has shown me. I would not be a Christian if I sought only what made me feel good. I know this is sounding a little strong but bear with me. I maried my W at God's request. From your post you entered your marriage for reasons other than God asking you to do so. Your H is working on getting his relationship right with Jesus just as you are. He is not perfect just like you. Yet, what you are doing is what some would call the holier than thou approach.
You have children to consider. You have to consider how this is going to affect them. From researchers who have been on Focus on the Family the effect is one of the worst things parents can do to children. I can attest to that because my W is the product of a broken home. Though my parents aren't divorced, they both committed adultery. It was devastating to me to find that my mother had committed adultery. I knew about my father as a child which is why I am faithful to my W and mostly because God has been directing my path.
Your marriage like mine began with a sin: fornication. My W and I had sex before we were married before man and God. I did so out of ignorance. But ignorance does not dismiss the act as being wrong. Sin only causes pain as you are well aware. Will your pain go away by divorcing you H? No, because the damage has been done. Only time, lots of prayer, and discussing with your H is going to heal those wounds.
I'm not telling not to divorce. But God wants you to reconcile with your H because he is asking for your forgiveness. If you truly love the LOrd, you will do His will. DOn't do it out of oigation. Do it because you truly love God. I see us all doing a lot of things because we want to feel good. We need to start doing things because we love God. When Jesus died on the cross for our sins, I am willing to bet it felt many times more painful than all of our pain combined here because what He felt included what we feel. That's love. What we refer to love pales in comparison. We are to follow His example.