I think there are two issues at play here.
(1) The AIs at the higher difficulty levels can rush buy ships/buildings and so keep their research values at high levels. This, in turn, leads to an advantage in researching discoveries faster than the player.
(2) I play with tech trading/stealing turned off but before I did this it appeared that they were forming coalitions in their research endeavors and thereby exacerbating this advantage.
I play at suicidal/obscene levels and this is how I handled this. I chose the technologist profile (+20% tech, +1 sensors) with a +30 tech for an ability. As I said before, turn off tech trading and tech stealing. It is important to note that I play Gigantic maps with rare stars/planets/colonizable planets mostly. In a map with more planets the Federalist profile works better (with the econ bonus).
IMHO