AGreed with Jorgen. While he and I differ in exact strategies (I will tend to upgrade a bit more than he does), I think what's really important is that you implement a simple rules based system for developing a good economy. Doesn't have to be perfectly optimised, but Does need to be functional and competitive with the player which it is not right now.
For example, when an AI colonies a world, it should decide up front a specialisation for it, based on planet bonuses, planet class, number of planets in the system, and finally tile bonuses. Basic logic is:
1) is this a single or multi planet system? How close is this planet to others? If close, it's a great candidate for a manu world, since shipyards can pool production from multiple world's. If solo, it's a great candidate for wealth or research world's - since these do not benefit from being near other world's
2) how many of each specialisation do I have already? The Ai should be aiming to roughly stick to a ratio between the specialisations, based on what victory condition they are currently aiming for and their race characteristics. Ie drengin conquest, altarian influence/tech, etc.
3) does the planet have any bonuses? ghost world etc.
4) what tile bonuses are present?
5) what class planet is it? I generally reserve high class planets for my primary and secondary specialisations. Ie, if I'm prioritising research, then I want to focus on research and prod (for expansion, military etc).
Based on those factors (and maybe more), the AI can then make a good specialisation decision.
Then it needs to choose what to build where. Ahead of time. Again, as mentioned, the AI should always build a few factories first with 100% production to speed up setup times.
When choosing, the AI actually has an advantage!!! It can plan out the world much better than a human, as it can.model where to place the terraforming improvements. Humans can't really do that since they can't see what tiles are unlocked until after they've researched the techs, usually in midgame.
Where there is a big cluster of tiles, build amplifying buildings (solar plant, etc) in the middle, then core buildings around the outside. Where the clusters are smaller, build non core buildings. Factories if not a manu world, farms, approvals. Again all clustered.
Just implementing that stuff alone would be huge. The AI would become so much more competitive.