I hope I'm missing something on this one but...

When i set a rally point for newly constructed ships to go to the first one will always follow instructions. However, once the second ship is built the first ship then leaves the rally point and meets up with one of the nearby fleets around the planet. This is a pain when trying to organize your fleets and requires me to have to micromanage each ship as it's built. Is there a workaround or fix for this? Maybe I'm missing a setting that I could change?