Does Malice trigger when a ship takes damage from the Vengeance ability? 

In other words, if a fleet of ships is attacking a target protected by Vengeance, and the fleet is currently affected by Malice, does a part of each ship's Vengeance damage get applied again as Malice damage to the entire fleet?

I could see this getting really ugly, really fast, considering that your fleet is also shooting the affected ships, and their damage would also be boosted by Malice.