You're looking at it backwards. It's not about women being smaller in general, it's about men being larger in general, this is due to hormones. Men grow larger bone structures to have advantage in the wild and hunting. Women didn't need that back in survival days. Now the difference is useless since we don't have to hunt and survive.
...probably ties into my intense primal desire to caress.
Becaaauuse I'm greeeeedy! HELL YEAH *hi-fives Wario emoji*