High level fighters are not normal folk. What exactly sets them apart from normal folk varies from character to character. They may come from some supernatural ancestry, or be chosen by the gods, or fate. They might have gained power through exposure to some magical energy. The rules don’t prescribe the means, so that players have the freedom to define it for themselves, as suits their concept.
It seems like I can call it whatever I want, as long as I want to call it magic. Because any time I dare suggest a character be capable of superhuman things by means other than magic, we end up having this same argument.
Of course that’s the case. Like, literally, that’s what high-level PC abilities do.
Getting enough XP (if your DM even uses XP; it seems like few do anymore) necessarily entails going on fantastical adventures, delving into mysterious and magical places, fighting monsters and faeries and perhaps even gods. One does not do such things and remain a humble farmboy. What exactly causes the transformation from farmboy to hero varies from story to story.