Well. Flavor they've put some work in allready... kingdoms turn the terrain around them green and lush, empires turn it into a twisted wasteland.
The story based difference is, the fallen were the followers of the titans.. not just by choice; the titans created them. The titans wanted to take the world's magic for their own.
That's not exactly a nice thing, but that doesn't make the fallen, in and of themselves, a bad people. However... death worshipping, brutal half-demons and such is what they are so 'bad' fits, and some technologies do emphasise this facet.
I think I'd take the flavor thing a bit further, namely with music differences and ambient sound (That might be done allready, but in that case, not thoroughly enough because I haven't noticed it.).
Now, the population structure leads to shantytowns.... that fits well with not caring much for general populace, but not so well with extolling skilled individuals. As such, a research line that lead to estates or similar - productive sites in one way or another, or even better, increasing the abilities of champions, should fit.
I'd love a faction - specific research line, like in galciv II. Wraiths would have magic-enhancing death rituals, Yithril could have combat arenas, and so on.
But slavery (in some forms), random torture and such, that seems more the domain of a caricature of evil; eebil. Although a colosseum style arena for prestige would work; it could increase prestige and income normally, and if you throw a bit of population into it, its bonuses would be doubled for 25-50 turns...