Ok, I know you've gotten a lot here, and a lot of it has sounded like people telling you to jump now. I can understand that. You might end up reading this as me telling you that too. But it's not exactly what I'm going to say here. I do think you are dealing with the same kind of questions that I was dealing with after DDay in my situation, and I'm hoping maybe I can help a little.
First, let me say - XH and I had no kids. We were married just under 10 years on DDay. But I'd known for several years before that that something was wrong, I just didn't know what, and couldn't get him to talk to me about it.
Dday was not really a surprise to me. I had started suspecting my WH of being a WH for at least 6 months or so. He had of course been denying it the whole time. When he told me about the A, I asked why he had told me, and he said it was because he had realized how wrong he was, and he wanted to fix things between us, and try to make our marriage work.
A little later, the OW called me. I didn't necessarily believe anything she said, but the fact that she called spoke volumes. When I confronted him, he denied it at first, but finally admitted what I already knew - he only told me because she was going to, and he was beating her to it.
Then came the months of him swearing that she had been the only one, and that it was over between them. During those months, little things would keep popping up that bothered me, that made me think he was not being truthful. But he always had answers for them all. So, I believed him. I loved him, and I wanted to believe him. And we had a lot of years invested together.
And then we had a second DDay. He was still in contact with her. I knew it, I just hadn't wanted to know it. But THIS time, he was really going to stop. Well, ok. So, we went on. We went to counseling. Right up until he walked out to be with her. Left a note that said in three different places that he really didn't think it was over between us, but he just had to go be with her to "find out."
At that point, I realized I had a decision to make. It was clear from the note that it was only a matter of time before he would show up wanting back in. And if I let him, I would have just established the pattern of the rest of my life. Because why would he try for anything more? I would have just shown him that he could walk out, do whatever he wanted with whomever he wanted, and when he was ready, come back home, and I would roll out the welcome mat.
Or, I could just turn him away. And then I would spend the rest of my life wondering if he could have turned it around, if maybe he really was sincere that time. Wondered if I gave up on our marriage too easily.
So I realized I needed a third way. I needed to provide him a way to come home that would prove to me he was sincere, that things had truly changed. So, I promised myself that when he called wanting to come home, no matter what he said, no matter how he begged and pleaded, my answer would be that we could try, but that he couldn't come home for at least a year. And during that year, he obviously couldn't continue to live with or see her, but he couldn't live with me either. And that we would have to go to marital counseling. And there would be no guarantees, that I was going to have to see some real effort from him during that year.
Within 2 1/2 months I had my answer. During that time, I uncovered evidence that he had been doing on-line personals for at least 3-4 years and maybe longer, that there had been at least 2 or 3 other women which, if not full-fledged physical affairs, were headed in that direction, and that he had never quit living with OW. He had been going to counseling that whole time with me, seeing one of our church ministers who is also a counselor, and lying to us both about where he was living. That told me all I needed to know about his character and whether or not the marriage could be saved, or was even worth saving.
------
I think your situation is similar in that your WH appears to be someone who may be hiding more than you think. I think that's what most of the people here who have posted cautioning you about reconciling with him have been concerned about. With some men, this sort of thing isn't a temporary abberation, but a way of life. And there are a few signs this may be the case with your WH.
You need to set the bar really high. And start doing a lot of looking around. Because sometimes, the guy we married doesn't really exist - he's just a facade that other guy is good at putting up, for a while anyway.