Yes. That happens when you left click on the individual ship and then right click on it's destination. Sometimes the destination that you're clicking on is the fleet itself not the location in space. Actually, dependent on *exactly* where you click you can get either the fleet or the point in space as the destination.

You can tell this by looking at the ship detail. It will say that the destination is Fleet 1234 or whatever. When this is the case and Fleet 1234 moves, the individual ships will "follow" the fleet. Actually it's kind of funny.

The best way to do what you're trying to do is to set a fleet rally point. You then can go into the governors in the civilization manager and set all your colony rally points to the fleet rally point you created. Then as all your planets produce ships you don't even have to touch any of them. They're automatically launched towards the rally point. The ships fleet up to your max logistics as they arrive. Then your ready to go kill people and break things, as you say.

This does kind of assume that you have all your planets building only the one type of ship. But if they're not and you have some planets bulding constructors or transports then what you can do is to create a different rally point and set the planets producing the different ships to send their ships to the different rally point.

Rally points used wisely can save you literally thousands of clicks in a long game.