I have one, and I don't use it as a weapon. OTOH, my pact blade is a rapier, so wielding the Pact Keeper's Rod in my off-hand is a non-issue in most cases.
Edited to add: This just occurred to me. If you're BladeLock, and you bond with a magic weapon as your pact blade, you gain proficiency with it. So... bonding a pact keeper's rod as your pact blade would grant proficiency with it, even if it were an "improvised weapon."
Okay, that's enough rules lawyering for one night. I feel dirty now.