Ok, let's have a hypothetical society where 80% of women give all their sexual attention to the most attractive 20% of men. When these women are pregnant should taxes from all men go towards their medical costs, their children's costs, etc? When these women retire who pays their pension? Surely not the taxes of the majority. If the answer is yes, then you are simply trading women as sexual currency to men as actual currency for which these men get absolutely no benefit.

Reproduction is at the core of our existence, we are programmed to want to do it. This 80 % of men aren't simply going to roll over and say "oh well, I guess I'll forgo sex and female attention".