It depends on the genre.
CoC does fine without an actual level base system. They reward you with skill progression and survival (although the latter isn't always a reward). Some other genres do fine with level based systems but I am sure they would be just as fine without.
In my own experience, whenever I have played a game that used a level based system like in D&D (for example), the focus ended up being on getting to the next level; a sort of "Are we there yet" mentality for RPGs. While the games that had no level based system, like CoC, we ended up focusing more on building the story and developing our characters. This may not be true for everyone, although I would encourage people to play an RPG that has no level based system and see how it goes.