C.R.
It's tough dealing with neighbors sometimes and that's w/o kids and being pregnant. I would knock on the door and talk to the mommy...lay all the blame on you and your pregnancy so they don't feel like they are being attacked and be honest(or as honest as you can be)...tell them that you like how well the kids play together but because you are pregnant and they are still so young you would like to limit it to once or twice a week...tell them that you will be putting up a pool but because of your current pregnancy you are not comfortable with their child swimming w/o them and then try to set a date where they can come over together...you can also mention that you haven't offered their son food because you are unaware if he has any allergies and you want to do the right thing by them(this will open the door for them to mention any religious restrictions as well). If they don't spend time with him in their own yard odds are they will decline your offer of an invite to play together in the pool and even if they don't they will now understand that they(and their son)need to be invited to come over. Don't stay in your house because you feel bad, it's unfortunate but by watching this little boy you are enabling the behavior to continue...tell him "I'm sorry but I don't feel well today, another day you can come over." Sun is one of the best things for depression(good old vitamin D)...so get out there and enjoy your daughter and the sun...and if you(or the little boy)have extra people over tell him no...and when you put up that pool...adjust the latch on your fence so it can't be opened by the kids, the inflatable pools are required by law to be securely fenced just like any other pool...the last thing you want is someone getting in and hurt. Best of luck! (and hey worse case scenario, if you offend them by going over there then they will avoid you and you'll be rid of the problem altogether.)