From a lore stand point, I want to know what all these little neutral colonies are.
I think it's implied that just about every planet is already occupied by some natives (mostly human settlers), and the planet militia are the local military force. The TEC, Advent, and Vasari are the only major empire builders, but there are many smaller nations they'd be fully willing to absorb. In that sense, I don't regard it so much as "colonization" as regard it is "subjugation".
If you read the manual, it implies that the colony frigates only have a few thousand people on board, mostly military personnel. This seems to reinforce my interpretation.