I decided to take a look at how much CPU utilization this game uses and I was amazed to see it's only pegging 1 out of 6 CPUs. 80-90% usage on one CPU, about 20% on another, and the rest stay relatively idle. This would most likely be a major cause for slowdowns on larger maps, or when your game has dragged on for some time.
Anyone else seeing this? Maybe someone with an Intel CPU could check to see what their CPU utilization is during the game.