There is evidence that this does indeed happen and that it has been quite a persistent dynamic. See for instance:
https://psmag.com/environment/17-to-1-reproductive-success
"Once upon a time, 4,000 to 8,000 years after humanity invented agriculture, something very strange happened to human...