I hate to say this, but there is nothing you're doing wrong. The upgrade system is borked, and has been for a while. This is old...
If you read the linked post, you will see 'how' the system is misbehaving. Speaking for myself, I have developed an instinct about working around the mess that the upgrade system is. Here are some pointers, but these are just rules that SEEM to work for me.
1. The game is stupid about comparing weapons. It will joyfully upgrade a lighting hammer to a warhammer, and a ranged fire staff to a two handed weapon, because it seem to ignore everything but base mundane damage when comparing weapons.
2. The game ignores what you have marked as 'upgradable' in the designer, and worse yet, what you have marked as NON-upgradable.
3. When upgrading armor, the game will completely ignore everything but base armor value, and will NOT try to account for weight, so it will ruin meticulously crafted designs.
So, does this mean that you cannot upgrade your troops. No. But it does mean that you will have to know what upgrade path you want your troops to follow from the beginning, and that you will have to upgrade at the exact right time.
I'm sorry, but all I'll give you here is some examples. I do not have access to the code, and it NEEDS CHANGING anyway, so this is just something that works for MY designs, for MY research order, in 1.02
Example 1: I build some early troops with clubs. They survive for a long time, get high levels, and I want to upgrade them to lightning hammers to take advantage of this. I HAVE to upgrade when I have researched "Enchantment" but BEFORE I research "Weaponry", or the game will upgrade them to non-magical hammers.
Example 2: I build some early troops that wear monk's robes and are designed for dodging. If I want to upgrade their shields (better dodging) I have to do it when I have researched "Blacksmithing", but before I have researched "Arcane Apparel", or the game will switch my dodging robes to non-dodging ones.
So... what can you take from all this? Mostly that you have to know WHAT you want to upgrade to, and make sure that you upgrade at the right time. Do not forget to make sure you have the carrying capacity, by the way.
Most of the problems would go away if the developers would fix the code that keeps track of what is marked as upgradable and what is not (the linked post talks about that) and make sure that weapon types do not upgrade into each other. But I have been posting about this since September, and all that got fixed was the exploits. Which may have been the right decision, given that those who upgrade eventually learn to work around the bugs.
