Oh boy...what a question. I have to go through the rundown.
Husband #1
We both seemed to have the same sex drive when we met. Basically all the time. Then, after having our first daughter, he almost completely lost interest in sex. Referred to his seeing me "as a mother now" whatever that means. In fact, we once went an entire year without sex. It never really came back. Our age difference might have been a factor, as well. I was 18 and he was 35 when we were married. We stayed together for 10 years.
Husband #2
Again, like most new relationships, we both wanted sex all the time. For the first few years, even though his drive was definitely much stronger than mine, I was able to show up most of time. But mitigating circumstances that I won't go into here took a definite toll on our relationship and by the time I left him 12 years later we were not having sex at all even though he was constantly pressuring & attempting to manipulate me for it. I had absolutely NO interest. Let me rephrase that, sometimes I would give in, but it wasn't sex to me - it was like I was just there, I felt nothing. Too much information? I think I oughta start a journal....lots of unpacked baggage with this one.
Current Flame
Well.
Now that I am past the dark days I meet someone who seems to be my ideal. He lives 250 miles away and I get to see him about once a month. He is HOT - by which I mean he is a great lover. Perfect for me anyways. Our sex drive and inclinations are perfectly matched. I feel as though I am experiencing a sexual renaissance...or I am feeling the consequences of stifling my sex drive for so many years. I don't know, I don't care. When we are together, he wants sex all the time and I want sex all the time. It works - I think I'm onto something here.
So how does all this relate to your question?
1) The sex drive can be low in both men and women. I think it is more common among men than is popularly recognized due to certain stigmas and stereotypes. Just as a low sex drive is popularly attributed to women because of stigmas and stereotypes. And you're right, it is cruel. And unfair.
But I can tell you that pressure doesn't help and can actually make it much worse. After all, there is little he can do about it. If it's not there, it's just not there. Having it pointed out can be very demoralizing for both men and women. I don't say that to infer that you are over-emphasizing it to him, only to point out that, if it's a sore spot, it will take very little attention on it to put him on the defensive.
2) Sex drives wax and wane with the course of a relationship. But if the relationship is sound, then it shouldn't be a problem. But, if there are exacerbating problems with the relationship it can become a major issue.
3) Maybe adding a little spice to the sex life might peak his interest a little bit more.