I read another post that said they wouldn't attack ships without weapons, but the poster may have been talking about peackeepers, it wasn't clear from the context. Is this true?
That would be the Peacekeepers, yes.
And they only don't attack ships without weapons until you attack them to destroy them-at that point they'll attack anything and everything they own.
Yet another reason not to arm your starbases, though.
You don't state specifically, but from your comment on 10 fleets, I assume this is the mega pirate event.
You might try turning mega events off. The normal "random" pirate event can be cleaned up by 4 or 5 ships, max. (Okay, maybe 10, spread out through the entire galaxy, if you're playing a gigantic.)