Just played a game as Yor and they aren't that bad. Like you said, you have to play differently, not use up all your tiles. But keepin tiles empty can cause you to have a bangin' economy (no maintenance cost for all your unused tiles). Combined w/good pop growth and the money comes in nicely to support a giant military. Some of the buildings have big effects for one building. Plus, smaller planets others don't care about are fine for you, you don't need many tiles, don't need to spam a bunch of low output buildings.
I disabled tech trading, so had no other econ techs, yet had the best econ in the game.
Harder in the later part of the early game I think as balancing morale/production/economy can get tricky till you get a good population, but once you have that population its all good.
I'd also add - don't upgrade structures till you are sure you can handle the increased costs, the newer ones are *much* more expensive. Lay off upgrading and go elsewhere in the tech tree.
And if you play w/random opponents like me, don't forget to change your color scheme and at least your symbol or like me you might run into the 'dark yor' and have trouble telling your planets/boundaries apart. Even the treaties screen works by symbol so you see yourself twice on it (iconians are selected. I'm the yor they are at war with, they are warm w/the other yor. Same symbol for both of us.)

All that said, I've only played 2 games so far. My experience might be unique to my settings. Immense maps both times and I haven't started ramping up the difficulty yet. But I had fun w/the yor just as they are.