Ideally, IMO, it'd be a variable you could set prior to the match, just like ship speed, fleet size, etc. 0/2/4/6/Unlimited was suggested in another thread and that sounds good to me. Perhaps give the TEC Loyalists a research following the Novalith that would allow them to build a few more at the cost of some tax on their income since maintaining a battery of experimental planet-nukes is probably not too dissimilar to maintaining a fleet in friendly territory.
As for the OP, the Novalith is annoying, but far from OP. The Vasari's Kostura is probably the most powerful, it just has to be used correctly, and you don't want to ever attack a front line world with it. (Well, I guess you can, but that's a waste of a shot.) It creates a phase node at the target planet, allowing the user to jump their fleet directly to that planet, let's say your capital planet. Since you've expanded since then, your main fleet won't be nearby and you'll have to watch in horror as your rear economic worlds fall. When they get done (or when you redirect forces to kill them), just fire up a phase node via an Antorak, Orky, Phase Gate, or another Kostura shot and the fleet can just jump out, having ravaged your economy. Novaliths can only take down tax income. Kosturas and the fleets they allow to invade can take down your trade ports which late-game should constitute most of your income anyways in addition to causing you to not only lose planets to neutrality, but lose them to the enemy.
The Deliverance Engine with a buff that would disable any broadcast centers on the target planet would be immensely powerful by allowing the Advent to attack the enemy within their own unopposed culture which drastically improves the survivability and damage output of an Advent fleet.