Until devs ditch the failed concept of having the AI play the same game as the human player (it can't), I don't think this problem is ever going to be fully solved.
AI can be written to perform fairly well in restricted environments (ie, tactical combat), but it's never going to be as good as a proficient human player on a strategic level.
It's even worse when the AI is expected to perform some sort of 'diplomacy', which also never works well, and is almost always hilariously open to player exploitation (or requires bandaids in the form of no tech trading et. al. to prevent weird problems).
I want to see a TBS dev tackle the entire construct of the game from the "AI"'s side of the fence by creating a challenging game world, and filling it with challenges for the player to overcome on the way to victory or defeat, instead of trying (and failing, over and over and over again) to create an AI that can play the same (ridiculously complicated freeform) strategic layer gameplay that the human can.
The players are somewhat guilty about causing part of this problem also - they will scream bloody murder if the AI 'cheats', and yet the player is possessed of innumerable advantages that the AI can never have, until we have some sort of revolutionary AI breakthrough.
The whole idea of having a level playing field with the AI needs to be tossed, because the playing field is not level, and I think it would be a lot more fun to play against a challenging strategy game than a challenging strategy game AI. Subtle difference perhaps, but an important one.
I think its a minor miracle that any of these absurdly complicated strategy games have AI that even approximates a human player, much less actually play the game well or present any meaningful challenge for a remotely experienced player. Insane variety of units (and god forbid, custom units, as is the case here), spells, equipment, heroes, buildings, terrain, etcetcetc, all add up to ridiculous numbers of options that must be coded around.
That isn't to say that these games aren't fun because of bad AI, more that they're typically fun in spite of said AI. The rest of the game can still be enjoyable for other reasons - art, music, story, exploration, building, powering up your empire/hero and conquering stuff, etc. etc.
Anyway. That's a whole rework of the game structure from the ground up, not something that is going to happen here. For my part, I've dropped the game and moved on to others for now. I hadn't touched it since beta early this year, and I was able to step all over it in my first game with the retail version. I didn't see the complete complacency issues reported here, it definitely had meaningful armies, both in its cities and on its heroes, but with the typical abuse of issues I had with the game in beta (hello teleportation), it was pretty easy to win.
There are some cool concepts in the game for sure, but I've found some other TBS games to scratch that itch that are either more challenging (Eador) or engaging (King's Bounty/Warlock's Armageddon expansion).