Hi B.,
There is no need to worry about your neighbor. Just don't hangout with them - period. You've got alot in your hands to take care of, and this is definitely not something you want to add in your daily busy routine.
When I moved here in 2000, suddenly I have all these "friends". Soon I realize who are my real friends and who are what I would just call "fiend". I wasn't happy with all the "drama" that I turned around and cut my ties with all of them - literally. Obviously, there's was more gossip and chat about me, but who really cares. There was no regrets and the next thing I know, I became so picky of who I want to be friends with. Unfortunately, it sucks!
Friends do not gossip and talk about each other, especially not behind your back!
No reason to tell them why you stop talking and hanging out with them, you just don't tell them anything. Remember, silent is the killer! As always, be polite, acknowledge each other when you drive by them, etc. Keep yourself busy and you will eventually "not" miss a thing about them.
Life is too short to be worrying about things that you are in control of doing and in your case, you CAN do something about it.
Good luck to you, don't worry, be happy!
-- R. D.