It seems to me that humans have varying degrees of agency, partly or strongly restricted by various structures (familial and social patterns and beliefs, and basic mental and emotional predilections, figure strongly here).
I've been attracted to the concept of "memes" and "Integral Psychology" as developed by thinkers/scientists such as Ken Wilber (http://en.wikipedia.org/wiki/Spiral_Dynamics). We all experience a conceptual/emotional reality that seems truest to us, and this model shows how our "meme," our understanding of reality, fits in the spiral of human development. Each meme transcends and includes previous memes, and each is absolutely essential to human progress.
The problem with this overview is that each of us will see that while our meme is a "higher" level of development than others, we are also lower than others, and this is hard for our egos to accept. We may even be convinced by our philosophy or brand of religion that our values are the highest possible, and be unwilling to consider the validity, and actual necessity, of other memes. Additionally, all memes exists simultaneously in modern society, which means there are a multitude of needs and motives that exist simultaneously, all pushing and pulling against each other.
This makes it awfully hard to state definitively that there are certain human rights that should be available to everyone. Different memes will understand different needs as more or less necessary, and will often fight fiercely for the validity of their particular views, rejecting all others.
I can even imagine, if everyone's opinions were graphed, that some people would stop with saying our only human right is that we draw a first breath, while others, depending on their take on reality, would argue for clean air & water, access to nourishing food, perhaps health care and meaningful work that pays a living wage. Lower memes might argue against our "right" to pursue happiness, or at least wish to carefully define other people's right to pursue happiness, particularly if they see life as basically a competitive exercise, with other people's successes costing them something that they value.
No matter how evolved or enlightened we are, it's extremely hard to accept that other people will define rights and needs differently. The right to carry firearms, for example, may be seen as absolutely central to some people's memes, but quite beside the point, or even counterproductive, according to other groups' versions of reality. The right of a developing fetus may be essential in some people's realities, while the rights of the woman who's already alive and grappling with her own needs is more important to others. A right to a loving partnership may be limited in some people's paradigms to members of the opposite sex, while others with different life experience may marvel that same-sex couples are not accepted as having the same rights.
I would love to see society grow enough emotionally to recognize the many ways that we are all connected, and understand that some people suffering unearned harm or injustice actually hurts the whole body of humanity. The most obvious way that occurs is that we all live with some fear of loss or deprivation as long as there are "in" groups and "out" groups. We do not want the risk of somehow falling into an "out" group.
When I was younger, I could almost believe that this last fact would be self-evident to MOST people during my lifetime. Now I'll be surprised if we actually get there in the next century or two. But for the sake of my loveable, beautiful and innocent grandson, and all the innocent children being born and raised today, I hope we as a species continue to learn from our errors and successes, and get there while there's still a chance at some wider sense of our interdependency.