Depends on the type of ship.  Many ships have their join fleet behavior enabled by default.  But not all.  Check by clicking on the fleet management button.  If the particular ship in question doesn't, then you have to highlight all the ships within a gravity well that you want to make a fleet out of and manually select create fleet. Remember that ships only check that behavior when they first enter a gravity well.  If a fleet later shows up to their location they won't join it unless you set it up manually.

It will only work if the ships have their join fleet behavior enabled, e.g. Cobalts or LRMs and the like, and the fleet is in another gravity well.  Then they'll join the fleet when they enter the new gravity well.  Otherwise, it will just move units to the flagship of the fleet.

I know certain ships will indeed just follow the flagship around if you click on the fleet icon itself.  But IIRC that ship might not attack or activate abilities while doing so.  It depends on the ship in question.  Also, if you just click on the flagship itself, the unit should just move up to it and then go idle until an autobehavior is triggered.  But it won't follow the flagship around.