Balance is whatever the DM throws at the party. If the DM doesn’t have to take into account the likelihood that the players don’t have the right tool they can use more powerful monsters.
There are occasions when you want to use it, but then you make the monster completely immune so the players have to go away and discover the McGuffin they need.
Fights in 5e are too short for players to trial and error the right damage type.
It’s also another way of punishing players for not playing casters, since casters have far more damage types at their finger tips.
In Call of the Netherdeep there is a mutant shark that can only be killed with radiant damage. There are no clues that it required radiant damage, and since there was no cleric, none of the characters did radiant damage anyway. In the end they managed to hit it with Hypnotic Pattern and run away, but I don’t think they found it a fun encounter. If they had had a paladin, the encounter would have been trivial, and they probably wouldn’t have noticed they needed radiant damage at all. I don’t think it’s a good idea to punish a party for not including the right classes.