I'd let you use an Unseen Servant in this way even if there weren't "rules" in place to allow it. In fact, the Servant wouldn't even be just barely passable, but actually a fairly good drummer at the beginning. And as the Bard leveled, the Servant would level up their drumming skill too...