I think I can possibly help you with that. To me, this looks like a pretty classic enlightenment vs romanticism issue.
The kind of fantasy you're interested in is rooted in a modern view of the world. In the reality of the 21st century, we know how physical laws work (to a point). We have data on what is and is not humanly possible, and even if you've never really looked at that data, living in a society that knows those things shapes how you think. This modern view is the basis for science fiction, super heroes, and some genres of fantasy--the world obeys the laws of physics, except FTL travel is possible because of advanced alien phlebotinum, or it's the world outside your window except some people are mutants or were been bitten by a radioactive spider, or society works just like you'd expect except there are vampires and other supernatural beings hiding in the shadows.
But one of the things one might want to turn to fantasy for is escape from exactly that modern viewpoint, to the way human beings used to see the world before the advent of modern science, where the world is defined by mystery and "wisdom" is more relevant then logic. And fantasy, traditionally, has filled that role: Tolkien is extremely explicit, for instance, that there is no division between natural and supernatural in Middle Earth. You won't find any such distinction in Narnia, in Lovecraft, in Edgar Rice Burroughs, or in Robert E. Howard. Part of the joys of these stories is precisely that they do not adhere to a modern viewpoint (except for Lovecraft, kind of) and that is one of the things DnD caters to.
But of course it isn't the only thing--Dnd is a mishmash of viewpoints, and some of them are extremely modern. The very concept of an "anti-magic field" is pure sci-fi, for instance. Arguably, Dnd being a relatively well-defined system does lend itself more to a "scientific" view of fantasy where magic is just alternate physics. And it's true that recent fantasy has tended more towards a modern direction, though this certainly isn't universal.
It sounds to me like a post-enlightenment viewpoint is so central to you that people looking for romanticism in their fantasy is actually incomprehensible. But humanity has only had a scientific conception of the world for a very, very short period of time, and wanting to harken back to a more, well, fantastical conception of the world is perfectly natural.