Why I like pioneers costing population:
It gives meaning to the population growth rate. It is an extra interesting reason to make a city a town, to build the well and subsequent growth buildings.
It reduces pioneer spammage - extra growth is something one really has to work for, or to wait for.
From an AI point of view, here are the times to build a pioneer when the cost population:
1 If your city has reached it's population cap, queue a pioneer so that it keeps growing (possibly keeping producing what is already queued.
2 If you have found somewhere to settle, and you need a pioneer, and your city is not about to level up, queue a pioneer.
3 If you have found somewhere that needs an outpost, and you need a pioneer, and your city is not about to level up, queue a pioneer.
4 If you are not producing pioneers fast enough, get a town and increase its growth. Or upgrade outposts so they provide growth.
Personally I don't like the idea of not choosing the more interesting cost, because it is hard for the AI. In my opinion, the interesting part of the game is the expansion, and conquest of the monsters. If the computer players are not very good, just cranking up the difficulty is an easy solution.
Suggestion - higher difficulty levels could have increased growth rate for AI players, if one cannot manage to get them to produce their pioneers strategically.