Well, the sheer randomness of the combat system is a little too much no matter who you are. I've been experimenting with different playstyles, I had a maxed-dex sovereign the other game with some absurd amount (like 130ish) of defense, and I winced every time some 15 attack bandit whacked him for 12.. it happens more often than you'd expect.
But it feels like the randomness hurts casters worse, due to limited mana. If your maxed-attack sovereign gets a bad roll and misses a 5 def enemy with his 100 attack, you can always swing again next turn, maybe you'll lose a few extra health due to the delay. If your maxed-defense sovereign gets a bad roll, you can lose a third of your health against an insultingly trivial enemy, but you can still finish him. If your caster gets a bad roll and misses a high level spell, you not only lose time and possibly health if your enemy attacks you back before you can cast again, you also lose a significant chunk of the very limited amount of total damage you can do before your mana runs out, and it doesn't take many bad rolls before suddenly you realize you can't finish this battle.
Honestly, rather than making spells into epic destroyers of armies, I'd rather see much more mana regen both in and out of combat, so a caster sovereign could practically match a melee hit-for-hit in a long battle, and repeat this performance every battle - it just seems more balanced. Misses would still be irritating, but not as crippling if you just had to wait an extra turn to get the mana back and try again. I mean, I don't want magic to lose its epicness - I wouldn't want casters turned into archers with fancier special effects - but I'd rather have a lot of sustainability (i.e. mana regen) than a little more power. What if wisdom granted mana regen in addition to the current effect?* It'd let players choose between more damage to wipe out armies with their initial mana pool (int), or the ability to cast continuously throughout every battle (wisdom).
Of course all aspects of combat, attack and defense and spells and everything, could do with a hefty dose of normalizing (i.e. making average results more common than extreme lows and highs), which would help everyone - but casters particularly, due to the reasons mentioned above (how much you have to count on those very limited spells not missing).
*Not even sure if the current stated effect of wisdom ("make spells cast for less action points") even works, but if it does, adding regen would make sense - casting more but not stronger spells lowers your mana efficiency, you'd need more mana to keep up with an int character if you focused on wisdom.