Suddenly Changing my pointer speed

I never had this issue before but not all of a sudden when I go from primary to secondary and then back it sets my mouse pointer speed to default its extremely frustrating. I like my mouse really fast and I made sure on both machines the speed matched yet still, once I go to my secondary (speed is fast) then back to my primary (speed is slow). Any idea why this would happen? Also, my secondary mouse speed will remain intact, it's only the primary that reverts to default, ill go back into mouse settings and set it to fastest and the very first time I drag it to my secondary it resets.