I'm surprised that the grandfather is allowed to be with, let alone live with, his grandson if he's a risk and on the sex offender registry. It might help to find out if the grandfather's offense involved children or adults - I don't know how the designations work in your state, but you can get that info from your local police department. You say "it was shared with you that…." but you don't say by whom -- so do get info direct from law enforcement. I don't know how much of the grandfather's case is public record, but police will at least be able to share what is public or about about the designation in general, so you are clear.
That said, I wouldn't allow my child to play at their house even if the grandfather was not considered a pedophile, because the grandchild has shown a level of awareness that he shouldn't have at his age. That could have come from the grandfather, but it could just as easily have come from someone else. If you allow them to play together, it's going to be in your house under your supervision at all times. Doors open, public play.
No one from that family is going to ask you "Why can't your son play at our house?" They know. If the kids ask, you say, I prefer that you play at our house. If you want to, we can go to the park/playground/beach." I don't think you need to explain beyond that.