There are many sides to this. I am very leery, in fact with some aspects that have made it out in public, outright disdainful of U.S. agencies utterly regarding the people of the United States to essentially be enemies and regarded as such in terms of taking way too many liberties in spying.
However, in regards to drones, the determination of right versus wrong isn't so clearcut. Pull back a bit from the question of "is it right or wrong for agencies in the United States to use unmanned drones to spy?" to more basic questions such as "who owns the air above private property?"
One might be tempted to declare that private land owners should own the air above their property, but such a basic declaration would utterly destroy all air travel, as air travel can only happen on the premise that the property owned by private property owners does not extend very far in the sky ... there are really no paths for aircraft to follow between cities in the U.S. that would not take those aircraft directly over private property. All of those aircraft have windows, and cameras and binoculars are not forbidden for passengers on aircraft, which leads to the reality: there is nothing at all to stop anyone from spying on anyone from the air in the U.S.
It is well understood that it is not only the authority of law enforcement, but indeed their duty to peer into suspicious activity regardless of whether the activity occurs on public or private property; in the olden days, it was policemen on foot or horseback looking out for crooks and criminals while patrolling. Even early on, it was realized law enforcement authorities must be allowed some "reasonable" level of authority to enter private property without express permission nor even a warrant on the grounds of "reasonable suspicion." "Reasonable"-ness is, of course, a vague descriptor ... if you were to define a restrictive standard for what is reasonable, what would it be?
In my home state where I currently reside, Washington state, there are some terrible wildfires burning out of control in the drier portion of the state east of the Cascade mountains. "Drone" aircraft are being used to help monitor the extent of the fires, which direction they are burning toward and how fast they are spreading which has triggered a number of debates. The smoke and dynamic nature of the fires means flights even by seasoned professional pilots trained to deal with emergencies can experience a conflux of circumstances that make flying around ongoing hazards, such as these wildfires, much more hazardous than normal flight. Is the government use of "drone" aircraft not just or reasonable?
What of the case of using drones to try and suss out terrorists or other armed and dangerous fugitives on the loose, such as Christopher Dorner (the former LAPD cop turned homicidal maniac who murdered the daughter and her fiance of an LAPD police captain whom had represented Dorner in a disciplinary case hearing [ http://en.wikipedia.org/wiki/Christopher_Dorner ])?
I'm certainly not going to argue that any and all use of drone aircraft is justified, nor that U.S. law enforcement and spy agencies have not overstepped their bounds, but I find it very frustrating when someone pursues a course of action based on a gross oversimplification without considering the arguments they are making in a broader context and taking even basic steps to ensure their arguments are logical, reasonable and sound.