D&D 5E "I am not magical, but I use magic." - The Mundane Mystics

[Dannyalcatraz]

The other casters would have similar "Achilles's heels".

Creativity is your friend.

In accord with my previous suggestion, you might look at the power disadvantages & limitations in toolbox systems.

You could limit Sorcerers (and any other class/race) who innately channel magic by making casting fatiguing.

Those who source their powers from divine beings could have their powers diminished or even cut off if they violate the tenets of the faith.

Those who were empowered by pacts might face similar consequences for violating their agreements & taboos. Or perhaps their powers are diminished/ineffective against certain targets or under certain conditions.

Certain magic might be easier for some casters than others- one who excel at weaponized magic might only be able to do protective/healing magic with long rituals involving specialized items, and vice versa.

 

[Vaalingrade]

That's fair. A lot of people aren't. I suppose I equate it to being a Fighter without weapons. You still play the character, but until you get your weapons back, you just aren't nearly as effective.

People seem to think it is ok to be able to cripple martials by removing armor, weapons, etc., but not taking magic away from casters.

While I hate that too, this is more debilitating than that. Take the fighter's weapons and they can still wale on you with a stick as an improvised weapon (or just shout 'club!' and it becomes a regular weapon) and is down a few bonuses. Take the caster's spells and... well they're not a caster anymore. That's why cantrips are infinite, not some desire for power.