Hi there,
If it were me,I'd put the toy it in my pocket or away out of sight. "I can see that both of you are disagreeing about what's going to happen with this toy, so I'm going to hold onto it now." Then no more talking, no more indulging the boy in his threat.
It wasn't miserly of you. It was your son's toy and not yours to give away. My only suggestion would be that if your son brings it up, maybe cue him in as to what you plan to do the next time this comes up. "I noticed that Bobby really wouldn't let go of the idea of taking your Lego home, and it upset you. Here's what I'm going to do the next time:if Bobby starts to tell you he's taking your toys home, we can just put the toy away until after he leaves. Let me know if you want me to help you boys find something else to do, too."
For what it's worth, young boys who love Legos can get pretty weird about Lego pieces, because they are unique and coveted. I have heard from more than one mom about playmates putting them in their pockets and taking them home. Another idea would be to put the LEGOs away when that child comes (somewhere out of sight) and just tell the child "oh, I put those away, and brought out X Y or Z. You can play with those."
And before the playdate starts, just review the rules. I do this with my son's playmates ("The door stays open. You may play on the bed, however, there is no getting in the bed, and there's no playing in the closet." Those are my three.) and then the expectation is set. Keep is short and keep it relatively positive. You did okay, given this was the first time it happened! :)