I meeeean... Raistlin was Charismatic AF. They just never show that on the character sheets because they're focused entirely on selling the idea of him being a Wizard, for starters.
Over and over he manipulated his companions and brother. Again and again he used his knowledge to manipulate others throughout his life. Heck, he stood out in a crowd with Gold Skin and Hourglass eyes, recognizable by tons of folks. In D&D terms that's all high Charisma.
Granted, he was good at Arcana checks but...
shrug Either way, really.
Though like
@Micah Sweet says, just use A5e of Intelligence Warlock or say the Fistandantilus Patron changes your spellcasting mod from Charisma to Intelligence if you really want to try and maintain things in that way.