I want to tag along on this too since I have this happen as well. Mine may be slightly different, but certainly similar vein, so I'll add what I've noticed.
I have noticed for myself that after a couple hours - usually with either larger maps or more empires, especially if TEC is involved with trade escorts and many garrison planets - the sim speed will start to slow down, accompanied with lower framerates (sub 20,  even sub 10 in worst case).
I notice during this time that my GPU (NVIDIA GeForce RTX 3070) AND my CPU (AMD FX 8350) utilization are on the lower end of their capacity.
Both hardware items do not appear to be overheating and under a throttle as temps are between 40 and 50 C, and yet both are not using processing headspace that they report to still have.
If I'm in a multiplayer game, this slower sim speed results in a real-time time delay that can be up to minutes - An empire getting destroyed, as confirmed by teammates over voice chat, then minutes later I get the notification in-game.
It's as though my hardware "gives up" and doesn't want to work anymore.
CPU utilization clocks are usually sitting 1.6 - 2.4GHz, when they should hit up to 4Ghz(stock).
The behavior of the cores appears that all 8 fire at about the same rates, rather than using a boosted single-core.
During earlier parts of the matches, my GPU  and CPU will be at a higher utilization and be a relatively smoother gameplay with decent enough framerates.
I understand that my hardware is older, to say the least, but it seems to me that my equipment should be pegged out with trying to keep up, not getting "lazy" as it were.
Hardware:
AMD FX-8350
NVIDIA GeForce RTX 3070 8GB
32 GB DDR3-1866
 
@Orion_1 - My GPU will drop to single digits, or 0% like your video has shown for you as well.