We have this issue also, I believe. I wouldn't call it 'lag', however. Here is what happens:
We play multi-player over a LAN only, we don't really play much single player (but it will happen there as well, it just doesn't seem to happen as fast). We almost always play very large maps, mostly the Huge (Multi) random, or one that we made in-game. I can't say for sure if it starts exactly after each of us gets 5 planets (for us 5 planets is usually pretty early in the game on those maps). But it ALWAYS happens and it gets worse as the game goes on.
It is not a framerate thing tho, we can pan around and look at things just fine, generally. The empire tree, however, stops updating (you can scroll but it doesn't draw anything, it just stays blank until it is done with whatever it is that it is doing), the build buttons don't respond (they do receive input, however, because usually when the game starts responding again it will do whatever the player clicked on. Resources stop counting up.
Now, the interesting thing here is that because we are pretty much always playing two-player LAN versus AI and we are sitting next to each other we can see each other's screens. Well, when it happens to one player (game stop) the other player is fine, and when it (game stop) happens to the other player the one player is fine. In other words, both players never seem to be halted at the exact same time. I can't be 100% sure that both players are NEVER halted at the same time, but whenever I am halted I look over at his screen (nothing much better to do) and he is always fine, and whenever he is complaining about his I am busy doing things (so mine is fine).
I am on a different computer right now so I can't send a save file or anything at this moment, but I will try to remember to do so. I game this on a Core2 Duo MacBook (with Windows XP SP2) and my friend uses an Alienware (AMD P4 equivalent, I believe, top spec when he got it but it is getting dated but I'm pretty sure it is single-core).
The length of the 'pause' and the frequency seem to increase as the game goes on. We have NEVER gotten any 'minidump' and it is very very rare that either of us crash to the desktop with this game.