It could be a bit of both. A new improvement and an upgrade.
I build a market at lvl 1.
When the city levels up, I can build a 2g shop over the 1g market, replacing it.
That would keep the clutter in check.
But the easiest implementation is of course, to create additional "Market" improvements - with the exact same look and function - that you could build for every city level.
Internally they'd have different IDs but on the build interface they'd all be a "Market". One for a lvl 1 requirement, the next for lvl 2...
I just picked the market at random. That would work the same way for material.
It's just... messy. You'd end up with lots of small bits of production so the afforementioned upgrade thing would be nicer. However, if you have
to find the market in a big city in order to build a better market on top of it, that wouldn't be fun.