As I posted in another thread (
https://www.enworld.org/threads/what-game-systems-do-you-have.699515/), I have close to 60 different game systems. I'm still looking for a system that will fit the unique needs of my group.
Can you help find my Goldilocks system?
- Survivable – you’re not going to die to a single hit from a kobold at 1st level [this cuts out most OSR systems]
- Interesting Options – you can do more than swing a sword or cast one spell (if you want to interact with the game that way) [this also cuts out most OSR systems]
- Easy to learn – you don’t need to perfect your tactics, count on your fingers to hit elevated numbers in the mid-20s (with always altering numbers) [Pathfinder, 4e, etc., are cut here]
- Good GM tools – encounter building that works, possibly good adventures/settings [cutting out 5e]
You are basically describing why regardless of what system I am playing I always end up homebrewing my own system.
My particular wants in a system are a bit different than yours, but I have the same basic problem that no system really covers it. So, regardless of which system I use, I always end up with enough house rules to write a sourcebook. I have over 600 pages of house rules for 3e D&D, and if I had the need and the time I'd probably be able to expand that out to at least 6000 pages - essentially an entire D&D system or edition lifecycle worth of material as a stand alone - nine or ten hardcovers and a dozen of the old 2e style softcovers that they no longer make. I'll never get there. I won't live long enough and I have other things to do that are more pressing and important to my life, but that is the scale of what would make me happy with a system. IMO, it would be the best edition of D&D ever - it's already the best RPG I've ever ran - and of course it would make tons of people unhappy for lots of reasons.
Similar things apply to BRP or D6 or Traveller (which is a system that has had multiple people like me try to do the very thing that I want to do with 3e and then publish it). Every system I ultimately want to rewrite because no matter how good the system is it doesn't perfectly fit what I want to do at the table.
The closest I've found to a system I don't want to rewrite is Pendragon, but that's only because neither I nor anyone else I know has the time to invest in playing it the way it's intended.
To me rules light systems are designed only for one shots or occasional casual gaming. They lack sufficient internal consistency and sufficient modelling of the universe to tell a longer story. They are like short stories told in worlds where you can ignore the protagonists lack of agency or the plot holes in the setting because you aren't staying there long. But even amongst rules light games I haven't really found many that are compelling. My favorite is the obscure Goblonia system that uses decks of cards as fortune mechanics, but even it I want to rewrite the guidelines on the skill system because I don't believe the designer really thought through the implications of his guidelines.
As for rules medium systems, which is I think where you want to be, my experience with all rules medium systems is that they are simply rules heavy systems that not enough supplements have been written for yet. In fact, I have this tongue in cheek saying that there is no such thing as a rules light system; there are only unsuccessful systems that there is insufficient demand in the market for new supplements. If you look into my writings about RPGs, you'll find that I believe that it is inherent to an RPG that it is a collection of minigames designed to resolve different tasks. In the real world, the expectations about task resolution are varied enough between tasks that no one minigame well simulates the expected results of the task even abstractly. Rules light systems tend to deal with this by ignoring the problem. You won't be playing them long enough or seriously enough for the problem to come up. Rules medium systems tend to deal with the problem by limiting expectations of the sorts of things you are expected to do in play. When those expectations get violated and you start to have game play loops that are outside of the conventional expectations of the system for what an adventure is then the rules medium system ends up morphing to a rules heavy system by gluing more minigames on to it. And if it doesn't do that, it's because no one is playing it very much and so no demand for variety of play ever develops or the problems in the system never stick out because no one table ever plays enough to get annoyed by the warts in the system (or they do and just quit the system in frustration).
The short of this too long to read post is that my guess is that you'll have to write your own system.