M.R.
If your gut says not to let her go, then bite that bullet and own your decision. You cannot soften your thoughts here to make them more palatable to other people who must not share your view point, and if you are going to have vastly different comfort levels from this family, then it is best that you both know where you stand. Be nice, but tell Mom what you think, I doubt that there is much you can do to be dipolmatic here.
Just a thought here though, if you trust your daughter's friend's Mother enough to let your daughter sleep at her home, and if the Mother (presumably) set up this sleep over at her parents home (with a pool, so there is a reason to hold this party at the G-parents house) then I would presume that this Mother (if she suspected that Gramps was a pedophile) would never, never subject her own child, let alone other children to a danger by planning a sleep over at her parents home. Unless you think that this Mother is in cahoots with her dad, and is bringing in his "victims" for him. If you have something real to fear, it is possible that she has no idea her dad is a pedophile, in which case, even if you meet the grandparents, you won't be able to tell either. If she can't tell, how will you? How are you going to tell if any man she may come into contact with is a danger? How far are you willing to go to protect her from a possiblity without evidence of higher than average probablity?
You have to go with what is comfortable for you, but you can't also have your cake and make everyone, including your daughter, happy about what they get to eat! Your daughter will be angry, her friend will be hurt and disapointed, the other Mother will be insulted, and grandma and grandpa will probably be blown away by your concern. If there will be other girls at this party, be prepared for them to discuss the reasons for your daughter's abscence, and for catty 14 year old gossip to go around about your child and get all tisted around. That is the worst case sceanero, but it is likely to happen to some degree, that is human nature. No matter how you try to state your concern, you can't change how human beings will react to some kinds of situations. You can level up with this Mom and tell her about your concern, but the more you try to hide your intent here, the worse the situation could be; I think that they will see right through any dipolmacy you try.
I have a 13 year old, and she has a group of friends. I know thier parents, and I trust them. I would not hesistate to allow my daughter to go with them if the parent I trusted set it up. I would take the risk that something bad could happen to my daughter under the circumstances you describe, but I think your risk is small here if you trust the Mother. Your call, but think ahead, and think about how people will interpret your fear, because there is not much you can do to change that.
M.