Guys are guys, & they seem to ALWAYS be thinking about sex!
That said, I would take this as perfectly harmless. Think of it as the adult version of Beevis & Butthead humor... There's a reason the demographic for that show was adolescent & young adult men.
Among my classmates, & previously in the workplace, people would say something, & someone else would say "oh, not going there!" & then everyone else would laugh as the double entendre was exposed. Usually from an innocently made comment.
If it were me, I wouldn't blink about this. Now... if it happened frequently, & the person on the receiving end was uncomfortable with that kind of humor, they would need to say so, tactfully. "Fred, sorry, that kind of stuff just isn't really funny, I'd appreciate it if you left out the 'double meaning' jokes around me". If it continued, then take it further.
Don't forget, being married doesn't mean being locked in a purity box! A ring on the finger doesn't mean you can't ever joke around, or compliment someone, or find attraction to another. It just means that it doesn't go further, because you are committed to your spouse.
So, long answer short - No, I wouldn't worry about it, or be bothered by it. For the wife of this person, if it truly is a big sticking point, then she just needs to let him know that it hurts her, & ask that he keep that "harmless" flirting reserved for her. T.