I figured out what this is.  Your huts are supposedly being upgraded.  Unless something is happening where the huts don't get upgraded, I think it's WAD though I don't like it. 

I am wondering if you just lose the huts and don't get houses in return though.  I didn't see any spike in housing that should have occured afterwards   Someone should doublecheck that- if that's true, then yeah, serious bug.

Suggested fix: huts get upgraded to houses immediately, or when the structures are being upgraded, they still count as huts.

Ah, the situation I have is the max pop lowers.

Only seems to happen for Kingdoms not Empires on the first housing tech, haven't experimented with anything else , just shanties and houses.