If you enjoy the evaluation of RTS AI then I suggest that you acquire a much older game, Age of Empires II and its expansion.  What is special about AoEII/AoC is that the scripting for its AI was made accessible to players in a simple format that could be altered with a text editor.  This allowed for players to script their own AI, so that the best AoEII/AoC AIs without cheating on resources at all became more of a challenge for the players.  The AIs could even be forced to fight each other, to determine which approach was best, and you can download some of these competition AIs.  Anyway, I suggest that you might struggle more against several custom (though non-cheating) AIs in AoEII/AoC than against stupid cheating AIs in other games.   You would have to fight AIs with entirely different approaches, rather than over-resourced AIs with exactly the same approach.  One further potential problem is that AoEII/AoC is so well-designed that it features alternate victory conditions- you might have to attack the AI to prevent it from winning without having to break your defence. 

Other popular games have also featured some access to their AI via a simple text editor, notably the Total War series, though AoEII/AoC is the most effective implementation that I am aware of.  Why designers should hardcode AI remains a mystery to me, it just prevents any further development- and AI, rather more than other custom content, is an area where the boost of player involvement is much needed.

I think designers should consider designing their games so that enthusiasts can easily make mods, custom maps, and custom AIs.  My advice might be, "Build a good platform with good content, but allow enthusiasts to really do the heavy lifting for you."  Fans will increase the value and longevity of your game at almost zero cost to your studio, but anyone who wants to be able to access that content has to buy the game.  Sadly, I fear that a new (consolization-driven) trend of downloadable content will make studios less amenable to custom content.  (Compare Neverwinter Nights, which had a ton of custom content and multiplayer servers and persistent universes to Dragon Age which has none of that.)

The good news on the AI front is that if you want a challenge that's greater than AI, you can always try it in online multiplayer against human opponents.

Yea only the stock AI but I'd heard and seen the customs fight. I was more talking about the level of polish it had.... it truly felt like every game was different... the huge maps, the number of diff strategies for handling diff civs.... all of it was very well designed.

Yea I own AOE3 too and I agree. They took away a lot of what made the original great. Less fun siege craft, smaller maps with fewer civs and less victory conditions sorta made it feel too repetitive. The card system was indeed the best thing to come from it (of course the visuals were stupendous too). If only they remade the second one... wouldn't take much convincing for me to go out and buy it.

To be honest:  I'd be satisfied if the AI just knew how to handle starbases.  Being unable to smash through defensive barriers is a major fault of many AI's, though I think any comparison with EA's creations is hardly favourable.  Generals onward, they had distinct quality issues with their work across all categories.

You misunderstand; my personal opinion is that the AI's EA was churning out were absolutely atrocious, so bad that I actually consider the comparison insulting to Sins.

It should be reasonably easy for the AI to handle starbases better based on a simple routine where it checks whether there is one in the gravwell and then either delays until it has sufficient anti-starbase ships to attack, or attacks elsewhere instead.  The ability to retreat damaged ships is even more critical when attacking starbases, and one element of the poor AI at present is that the AI seems to attack starbases until a fleet is practically destroyed, perhaps there is a flaw in the retreat system.

Getting the AI to build its structures in an effective manner might be harder.  The AI has no ability at relations at the moment, as great a drawback as its starbase issues.

I just wonder whether the AI has any regard for starbases at all, or whether to the AI a starbase is no different from a turret.  I've seen AI fleets retreat from starbases when there's an enemy fleet present.  A crude fix might involve nothing more than assigning a starbase a combat value, perhaps?  Or have a combat value for all defensive structures?

I still feel that the Age of Empires/Conquerors AIs are the best ever produced for RTS.  Having a discussion on the worst RTS AIs ever might be amusing, but it won't get us very far.  The reasons I would advance for AoEII/AoC having the best AI is that the custom AIs produced by players can achieve what should be the aim of any good RTS AI designer, better play by better scripting rather than cheating.  The custom player-built AIs are often the equivalent of another level of difficulty beyond the standard AI.

This seems to have been achieved by having a means of implementing a custom script that is readily accessible to players, so that the player doesn't have to fight the interface in order to create a script , and by allowing the AI to fight among themselves, to allow for a survival of the fittest.  I've not encountered anything quite like it anywhere else- it remains a sort of El Dorado of RTS AI, or relics of the wisdom of the ancients, in more appropriate space game terms.

Here are some links: http://aok.heavengames.com/cgi-bin/aokcgi/display.cgi?action=st&fn=28&tn=39043&f=28,,,10&st=1450 for an AI ladder and http://forums.aiscripters.com/ for 'the home of ai scripting'.  If anyone knows of any AI communities with anything similar to this for other games, can they let on?  I'd like to be able to do a comparison, but I haven't found any so far.

The post Darvin made in another thread, about how it would be great to fight AIs that use different strategies, might be hard for an independent developer to realise, but if it were made possible for players to add value to the game with their own scripted AI then it is very achievable.  Perhaps even if it couldn't be done for Sins the feature might be a valuable inclusion for new games?   

Another retreat concept that might usefully be added to the AI is to have individual ships to ignore fleet orders and retreat to a friendly repair bay when they are damaged beyond, say, 50% level.  This would make it harder to finish off AI ships and so to fight the AI in general- and might even save some of the ships.  The ships could be made available for fleet use again when healed.

It is interesting and instructive to contemplate what sort of AI you would want to produce given the opportunity.  I know that I would cease any map-making work to concentrate on the AI, my opinion is that a game would be best with a limited number of maps, if this can then lead to improved AI and play.  Any AI I produced would build repair bays, if the AI had some means for its ships to make use of them.  Having them placed near appropriate structures might be harder to program or to make possible.

The existing AI might not be the worst in RTS, though the normal non-cheating AI is fairly dire, but it could be important to note all the ways in which the Sins AI has it easy:

-Pathfinding is made extremely simple, as the game is based on gravwells linked by phase lanes and has virtually no terrain or collision problems or stacking limits.  One of the game concepts is that antimatter is removed by travel rather than consumed by travel.  This makes the AI much simpler.

-Mining is automatic once the AI has acquired a planet.  Resources are infinite, so there is no end-game.  There are a limited number of goals, in terms of planets, for the AI to pursue.      

-The AI should not have build-a-group problems, because the unit limits are set so high.  If it does, it is due to the inability to develop its economy rather than built-in limitations.  There are no transportation units in Sins.

-Combat research is not terribly important in Sins, so that it is not important if it is done in a haphazard manner.  If each iteration of combat research yielded more traditional levels of improvement the AI might suffer. 

-It should be much easier for the AI to define the concept of 'territory,' based on gravwells, than in other RTS games.

I'm sure there are more, but just given the advantages I've mentioned the Sins AI should really be much stronger than it is at the normal level.  The implementation of the viciously cheating AIs rather than the further improvement of the basic AI was a very lazy form of improving the AI.  Though there were improvements made, the corrections made to the AIs early obsession with siege frigates were long overdue.  Of course the worst element of all is the the Sins AI remained hardcoded, which for a game where the AI environment is simplified so much and which is supposed to be very moddable, is close to unacceptable, so much that I don't understand why the problems with the 2GB memory limit have been emphasised over the far greater constraint of rigid and stupid AI.