And yet, you and I are able to talk about it because we did see and, to one degree or another, understand it. Which was my point.
That "bay leaf" analogy really does a lot of work here. Red from Overly Sarcastic Productions put it quite well in her "Planet of Hats" video:
"Showing us a fantasy or sci-fi culture in terms of the big differences between it and the familiar culture we presumably started in can be an incredibly useful tool to put us into the head of the main character, who's going to be looking at this alien culture with a buttload of preconceived notions and received homogeneity. But, unfortunately, this only really works okay if the culture you write actually has more depth than just what the main character sees at first glance. You don't even have to elaborate it in-story if you don't want to or it wouldn't fit. You just have to world-build it behind the scenes to make sure it actually holds together."
Again, I am NOT saying that it is impossible to go overboard. It totally is. It's totally possible to do a whole bunch of truly pointless filigree just because the author finds it enjoyable to do so. What I am saying is that a lot of genuinely BAD writing comes from failing to do enough "behind the scenes" world-building for things to make sense. Exclusively making loose sketches and papering over inconsistencies every time they crop up leads to things that make no sense or have ridiculous, unbelievable "solutions." Problems creep in that could have been nipped in the bud with just a little bit of world-building first.
So don't go to either extreme. Don't treat world-building as a self-indulgent exercise that never produces anything worthwhile to anyone but the DM. And yet also, don't treat world-building as this absolute requirement down to the finest details because you really will be mostly doing it for yourself.
World-building is NOT "completely unnecessary for a game of D&D." But some of the things people do for worldbuilding are unnecessary. Find the right balance point between the extremes of deficiency and excess--a balance point which will vary from one game to the next. For example, part of the reason I've done as much sociocultural and economic world-building as I have in Jewel of the Desert is that I have a physical anthropologist player. They're constantly asking questions about things you're deriding as "completely unnecessary." Food, building materials, etiquette rules, linguistic quirks, ceremonies and beliefs, clothing (that's a big one, because the player's character comes from a family of tailors), you name it, sociocultural stuff gets discussed on the regular. Meanwhile, someone like the OP, whose expertise is business, would likely ask questions about economics, trade, financing, resources, etc. A player with a medical career will think about medical concerns. Etc.
World-building is, and always will be, in part a matter of taste. Find the right balance point between "how does this even make sense? It's the biggest city in the world with NO FOOD!" and "why did you write three whole novels about it if nothing in those novels is ever relevant to us?" That will depend on you, your players, and the nature of the game you're playing (a beer-and-pretzels West Marches game probably won't need as much world-building as, say, a game where the DM has sutured together D&D 4e and a politics-focused PbtA system.) It's neither utterly essential nor "completely unnecessary."
Edit: Additional response.
People travelled more than you give credit. Many, many relatively ordinary folk went off to the Crusades, remember, and pilgrimage was something many tried to do as much as they possibly could. Yes, travel the way we travel now was not really a thing. But to characterize it as the vast majority of people never going more than a few miles form their place of birth? Inaccurate at best and outright wrong at worst. It's not like Roman farmers didn't tend to stay tied to the land they farmed.
That said, I don't disagree with your approach--so long as it really well and truly is the case that there's absolutely nothing of interest beyond the nation's borders. The problem is, I don't think that's true!
Instead, I think it best to treat it like rings. The innermost ring is the main settlements of the local area, the proverbial "starting town" as it were, or any place the party sets up as "home base." Such places will be interacted with frequently and often to a pretty deep degree, so they require more flesh on the bones. The second ring is the surrounding environs of those places, the wilderness and trade routes that the party will travel through frequently but not necessarily spend too much time in. Those require attention too, but not as much detail. Third, for lack of a better term, the "local periphery": nearby cities they might visit now and then, neighboring countries or city-states, etc. Such areas are close enough that their influence is likely to be felt, there will be trade and contribution, but it doesn't need too much. Fourth, the distant lands, close enough that you know that they exist but leaving them shrouded in rumor and mystery. And then, finally, you have the Outer Reaches, which is both lands known only through myth and rumor (which may or may not be true), and the genuine Beyond The Horizon stuff that, whether it exists or not, is too far away to have any influence on the story yet.
One might name them such:
1: Local
2: Regional
3: Peripheral
4: Far, Far Away
5: Terra Incognita
Apportion appropriate world-building to each. Local is your top priority, the details matter there. Regional, details might matter, so it's usually good to be prepared, and stay flexible. Peripheral will matter some of the time, but you probably don't need a ton of depth--just make sure it passes a smell test. Far, Far Away is distant enough that you can keep it vague and probably not have any problems. Anything further than that, don't bother--if it comes up, you'll have the freedom to do as you like and patch up any gaps later.