I think if you give this site a good review, you'll see tons of stories similar to yours. Certainly, anyone on either side of these ugly triangles can tell you that the fantasy is strong, but the reality is weak.
I'd suggest reading every word of the MB site, then reevaluating where your relationship is. I'm the spouse of a woman who had a couple of affairs. I also had affairs. I can tell you from my experience and what she's told me that those feelings you have for the other person are clouded in myth. I don't want to be a doomsayer, but I'm willing to bet once you leave your husband for this person and begin living a real life with real life worries, commitments and stresses, it will fracture and you will find yourself regretting the move.
If nothing else, you owe it to yourself and your spouse to see if you can try again. You don't have to go it alone. You can find a counselor, use this board, read books, talk to others. You have to find the root cause for the dysfunction in your relationship. Once you get there, you can determine if the plant can be saved.
I tend to see it optimistically. It can be saved, if for no other reason than you have a history together no "other person" can overcome. It will always be there, and there will come a time when you will wish it were still there.
The temptation here is to say "Well, this person can say that, but he/she hasn't walked a mile in my shoes." Yes, we have. Many miles with many more to go.
It's not a bold statement to say that the people who are trying to make their relationships work are all trying to make themselves better people. In the end, if the relationship crashes, they will still have that part of them to fall back on. At least they tried.
Is your fantasy doomed? From my perspective, I can tell you I know now that all my relationships outside the marriage -- and those of my WS's -- were doomed. We both know that. My WS said it yesterday: "After something like this, you see things with stunning clarity."
Good luck.
