Use more limited magic immunity. It's right there under your noses and no one looks at it when considering full casters as OP.
Creatures that have the ability where 5th level or lower spells don't affect them. This one really hurts multiclass casters because it doesn't matter if you upcast it, if the base spell level is 5th or lower, IT DOESN'T WORK AT ALL, and that includes cantrips- so that 4d10 cantrip looks a lot worse than that (1d10+5)x3 or 4 from the martials, now, huh? Even a 20th level wizard only has 6 spell slots above level 5. Most of these creatures have legendary resistances, and since a lot of higher level spells are "save or suck" (the golden standard for damage, disintegrate, for example), you're chunking off even more resources by making them waste the precious few spells they have that will actually work on the enemy.
The canoptic golem from candlekeep does it even better:
"Limited Spell Immunity. The golem automatically succeeds on saving throws against spells of 7th level or lower, and the attack rolls of such spells always miss it."
Now, there are plenty of below level seven spells that will still do half damage, sure, but any caster going up against this creature is gonna have a bad day.
Homebrew this a little based on your party's level. Have monsters that are immune to 1st, 2nd, 3rd, etc and lower spells.
MAGIC RESISTANCE BOTH TO SAVING THROWS AND DAMAGE ALSO EXISTS!!!!!!
Do some homebrewing of your monsters. We all know that D&D's CR rating is worthless, and there's no good way to stick a number to the difficulty of creatures vs your players, before shenanigans are even had. Design encounters with intelligent monsters with intelligent tactics that single out your casters, whether they're damage dealers or healers. Design monsters with the intent of hindering or countering your player characters, be they martial or casters. I had a barbarian that was buffed to hit for 40+ an attack with no exhaustible resources, what did I do? I designed a creature who's AC went up by 1 every time it took 30 damage or more in a single hit, and damaged those who hit it with melee. A cleric that excelled in single target damage? Ended up fighting a creature that reflected 30% of his spells back at him, and absorbed another 20% for random buffs. The bard won't stop enchanting people? Her victims complained to the local lord who was forced to put a stop to her rampant manipulation and subversion of the law and economy. EXERCISE SOME CREATIVITY.
Not creative enough to think of this stuff yourself?
Remember this:
ANYTHING YOUR PLAYERS DO, YOU CAN DO TOO
Give them a taste of their own medicine (responsibly, don't go TPKing the knuckleheads)
In a world where mages driven mad with power aren't all that uncommon, WHERE ARE YOUR ANTI-MAGIC TROOPS? Certainly someone with more brain cells than it takes to make a spark by rubbing them together has come up with the totally unheard of idea of DIRECTLY countering the problem instead of whinging about it until Bardy McHumpsalot seduces him into giving away his kingdom with ensorcellment or Whizbang the Infinite Fireball reduces it to rubble with.... wait for it.... earthquake (He's not a wizard, he's a sorcerer who's deceived everyone into THINKING he's a wizard. The name helps, too). Is it insane to think that in a world rife with magic that there would be specialized fighting groups, whether another adventuring party, a guild of protectors, or an army from the local lord/king/realm/etc that are specialized in the nullification, subjugation, and elimination of rogue magic users?
And hey!
Here's a novel idea: Buff your martials instead of nerfing your casters. Don't fall for the crab pot fallacy, where martial classes and subclasses CAN'T be buffed because that would make all the other martial classes look bad (like crabs pulling down any other crab who attempts to climb out of the crab pot, it's a thing), compare them not to other martials but to the casters you see as so OP. Level the playing field the right way, because I can tell you, having a nerf-first mentality is a race to the bottom, and no fun at all. Look at any game that embraces nerf-first tactics and see how their players complain that it's now less fun. You finally figure out a way to nerf casters solely by nerfing the classes themselves? Congratulations! Now they're just as fun to play as the martial classes you hate playing because they're so underpowered.
What a fun game guys, aren't we all having fun being powerless peasants? NO?!?!?!?!?!?!?!?!?!?! Guess we have to nerf you guys more until you have fun again.
AND DEAR FRIGGIN CHRIST PLEASE DO NOT INSTITUTE ANYTHING THAT MAKES CASTING MORE COMPLICATED.
Tier lists, randomized spells, rolling tables, specialized resource management - Stop it. Get help.
The game is complicated enough. Hell, TONS of players have all these kinds of issues with balancing because they don't read the source material enough. Don't make up more rules and complicated systems we have to learn on top of a system that's so broad and complicated already that half the arguments are how to correctly apply, interpret, or deliberately misinterpret the rules. I've been playing 5e since it came out and I still don't know all the rules and minutia. Nobody does. If more complicated is more fun to you, who am I to judge? But I can pretty much guarantee that you're the vocal minority on that point.
Heck, take OP's idea, just simplify it, no rolling or charts or certain spells have additional effects or not. The more powerful a magic caster you become, the more entities of all kinds take notice of you and want to use or eliminate you because of your increased magical powers. Could be a local lord that sees you as a threat to their power. A rival wizard thinks you're stealing his proprietary spell and has decided to do something about it. A dark mage thinks you're on to them and has to kill you to cover their tracks. And eldritch god has decided you're a perfect vessel for their resurrection. A passing mindflayer thinks your brain would be the perfect delicacy. The possibilities are literally endless.
And when all else fails? Lvl 9 counterspell.
