distraction, distraction, distraction. there's two things going on- first is that he is having a good time being out and doesn't want to go, second is that he's spent however much time being good and sort of on guard, but once you show up, he feels safe enough to let his guard down- this translates to meltdowns in little ones. it is very common and you are not alone, though I know it feels that way!
the key is to make it a non-issue by helping him transition to the next step- car, home, store, whatever. so to start, when you pick him up from grandma's, have something cool to show him- a new book or toy. but don't just run in and show it. go in, spend a little time. sit down and ask him what he did, what he played with, was grandma silly..... you get the idea. find a bridge, say he says "we played with dinosaurs". you say, starting to stand up, "that's so funny, I just got a dinosaur book form the library....." gathering his things, "will you look at it and tell me if you see the same ones you played with?"....kissing grandma, bringing out the book....."I saw one that was big and green, with sharp teeth, do you think you can spot that one?" moving out the door... and on like that. You are saying next to nothing about leaving, you are getting him interested in something else and focusing on other things. Clue grandma in on this as well. My mom was always trying to sweet and hugging my DD, going on and on about how she'll miss her too and she'll be back, etc., this just riles them up. the key is to focus on something else.
Now I will say that he's already got himself a nice little habit going, so this won't work instantly, you are undoing other behavior. But don't give up, it will get better. good luck!