If you think they are bad, wait for "neutral systems" (see under the map editor). They are fortified worlds- tough nuts to crack without a good fleet.
Oh yes, I love that setting. currently playing a slow speed ~400 planet map, 10 stars, phase lane chance set to minimum, phase lane length set to maximum, with neutral colonies set to maximum everywhere. 95% of the colonizable planets are fortified, with phase jumps between planets taking from 15-45 minutes per jump. scouting becomes critical before trying to expand into a system, as does the build up of a fleet that can hold its own without finding itself in urgent need of reinforcements, because there's no way to get urgent reinforcements anywhere, though at least the vasari phase stabilizers help somewhat with that, with the 40% phase jump speed bonus.
Seeing neutral fleets of 8-10 ships, supported by 3-5 gauss gun turrets, and 1-3 hangar bays, it's such a warm and fuzzy feeling, exterminating all those native inferiors who dare to try and obstruct the expansion of the empire!