This is how I see it going down ...
Horse: +3 movement, +2 combat speed, -2 bow attack
Horse with Saddle: +4 movement, +3 combat speed, -1 bow attack
Horse with Saddle and Stirrup: +4 movement, +4 combat speed, +1 attack
Units with a Horse should be able to buy a Saddle, and units with Horse and Saddle should be able to buy Stirrups.
As well, Saddle and Stirrups should be *special equipment* like a seige ram, but for Horse using units only. And you can't get a Stirrups without a Saddle. Perhaps some small gold cost for each (or materials).
Now, the horse should be the most expensive thing (medium gold +1 horse), while the Saddle and Stirrup are VERY cheap in comparison.
Also, I should think that "Saddles" and "Stirrups" each be their own technology, most likely requiring Horsebackriding and possibly one other tech. For instance, Saddles could require Horsebackriding + Leather Armor, while Stirrups could require Horsebackriding + Slings or Archery.