Men do not refuse sex. Are you waiting for them to approach you? How does it become infrequent?
I also agree that men DO refuse sex. Our sex life started before we were married...something I no longer agree with and intend not to do should I ever be fortunate to find someone else. In our teens when we were dating, it was 2-3 times per week. We were drunk. He was into kinky things. That was that.
We got married. Things changed. The long road of his resentment toward me, his increasing addictions to booze and to harder and harder porn happened, and during the last few years of our marriage, we had SF anywhere from once every 3 months to as far between as once every 9 months.
I tried. I told him I needed it. I tried to get him in the mood during the evenings, with touch and the like. But he purposely avoided me. He would avoid showering after coming home from his job as an auto mechanic. He would stay up watching TV, refusing to come to bed, until the wee hours, or he'd play his PS2, and then he'd crash and you couldn't wake him.
Or I'd try to wake him, and he'd grumble about how he had to work, or he'd curse at me and turn over so flat on his stomach I couldn't touch anything.
Yet he'd lock himself in the bedroom for hours from time to time viewing his very gross, hard-core, foreign porn complete with animals. He tried to show it to me a couple times. It was repulsive, and made me feel like I was nothing to him.
We are now DV. Test-driving before marriage did nothing but harm. Yes, when we DID have SF, he was very talented physically. But that wasn't enough to save things. Porn was easier for him, required no emotional commitment, and was apparently more exciting than me.
Given a second chance, I would take my chances and hold out until marriage, and I would make sure porn wasn't a factor. I firmly believe that if two people truly love each other for the right reasons, and if they've both been honest about being physically able to function, that their sex life will be just fine. I don't believe they will be incompatable.
Honesty and trust and respect for one another are the keys in my opinion. Not making sure you are sexually compatable before marriage.