My kids stayed with family (ie: the in laws, my parents) pretty early on. However, with friends, I think it totally depends on the child and who they want a sleepover with. I would say knowing the family is huge. I wouldn't send my kids to a home in which I knew nothing about the family. You can never be too careful these days.
My daughter is extremely independent and social. So she had her first sleepover with a friend at age five - the friend is a daughter of one of my husband's best friends from High school, so needless to say we know them very well - I'd call them family friends. With her school friends she was in first grade and again it was family we know well and the little girl is her best friend now.
For my son, he has yet to have a sleepover and is turning 6 this week. He is much more of a homebody, likes his sleep and to do his own thing. He just started asking for a sleepover about a month ago. As it turns out he was invited to his first one this week. Although it's a school friend, we've known the family for five years because the family's older son is in the same class as my daughter who is now in 3rd grade. I know he'll be in a safe envirnment and I think he needs to take this step of independence. If by some chance he calls and wants to come home, so be it and I'll go get him and we'll try again in one year.
I think ultimately you have to do what's best for your family. I'm a fan of telling my kids "everyone's family has different rules and we must respect them." My daughter's best friend couldn't spend the night at our house until she was 7, but my daughter spent the night at her house when she was 6 - for that family all of the family's kids (6 of them) had to wait until age 7 for sleepovers - house rule. Another friend says "no sleepovers until age 9" but she makes exceptions for parents she knows well like me.
If you are not comfortable with it, don't do it - do what works for your family.