I think your expectations about how much of the situation your 3 year old took in may be unrealistic. Most children that young don't really grasp the concept of death unless they see it - a pet dies or they find a dead bird in the yard - and we, as parents, usually overreact to their seeming obsessiveness and in the case of a pet will try to buy a replacement to make it better or if it's, as I mentioned, an unknown animal like a bird in the yard we become upset with what we see as their "morbid" curiosity. I'm assuming that your son didn't get to see his grandfather in the hospital and that if there was a viewing you would not have taken him given how young he is. This makes death an abstract to your child - it's an idea with no physical evidence to support except that his grandfather is no longer around. You, on the other hand, were able to go to the hospital and I would assume were present for all the usual ceremonies associated with a persons passing which gave you several opportunities to come to terms with it and help you begin the grieving process. Your son misses seeing and spending time with his grandfather - don't doubt that - but unless your father-in-law lived with you and had specific places/items that were only used by him in your home (bedroom, easy chair) which are now "empty", your son will probably only show signs of missing him when something or someone reminds him that there was this other special person in his life who is no longer there, that's when I agree a scrapbook that he can help you with would be a good idea. I also think you should make putting it together a fun and happy time - for those things that were special between your son and his grandfather that there are no photos of, take him to the craft store and look for stickers depicting those things and then maybe write a description of why it was important or what made it special. The DVD idea is good too, but I don't think your son would be able to appreciate it for a few more years. Since you are worried your son may associate a hospital only with bad things like sickness and death (you and your husband were probably very stressed or upset every time you went to see his father and even if you tried to lighten it up for your son, kids pick up on these things), you might want to find books that explain how hospitals work and discuss how doctors and nurses are good helpers when you're sick or injured such as "Clifford Visits the Hospital" or "Franklin goes to the hospital" - or my children's favorite - Richard Scarry's "Busy Busy Town" which just makes the hospital another interesting place to visit with helpful doctors and nurses along with the post office, gas station, etc.
My oldest was 4 when my father-in-law passed away. He had been ill for years and cared for at home. We live in a different state and she had only seen him a few times but was aware of who he was, so when we went back for the funeral and she walked into the bedroom and he and the hospital bed, oxygen tanks, tray of medication, etc. were gone, she had a pretty good idea that he hadn't just gone to run an errand or visit someone. At 13 she doesn't remember many specifics about him which may be a good thing because his health was failing fast by the time she was born, but she has photos of his entire life which included some things that were before our time like when he was in the Army, wedding, as a new father, a farmer, and then there are the things we were there for and we include photos of those - even some showing him in failing health, but most of all we are able to share our memories of him.