Ok, so map set up is:
Tiny, everything abundant, systems in tight sectors and very fast research with research vic off. And also those three 3 options checked.
Maybe it's because I'm not getting military units out fast enough. I really have no idea why, but so far, I've gotten my but whooped twice. My current starting build is 2 factories, 3 Xeno Labs, 1 Happy buidling, and the rest of the sqaures are Economy. Then my 2nd planet is straight Xeno Labs.