As a former lifeguard and Water Safety Instructor I have to say that you should not allow anyone, and I mean your kids too, in the pool unless an adult is present. The adult must be at poolside at all times when the kids are in the pool. I would suggest calling the parents of the kids in the neighborhood and explaining that the pool is not open unless they know that you, your husband or another pre-arranged adult is going to be there by the pool acting as lifeguard. This doesn't mean that they should assume you are going to be watching the pool just because you are home with the baby. It also doesn't mean that if someone calls and wants to swim that you are obligated to say yes, especially if you had no intentions of being out by the pool at that time. You don't want the other parents just sending their kids over whenever the kid wants to swim.
I would also not allow kids to come over with their parents whenever they felt like it. This is your pool, in your yard, not a public swimming pool. You are liable whether or not their parent is there.
Explain to your kids and their friends and parents that this is a safety and liability issue and that as much as you'd like them all to be able to come over whenever they want to use the pool they just can't. Your kids could call their friends when the pool is open and available or you could have a flag that you put in the front yard so the friends know when it's ok to come over.(Just make sure to take it down when it's not pool time.) Or there could be some other signal your kids could have with their friends.
I would definately ask the kids what their swimming ability is. Appropriate PFDs should be worn by those who need them. These should not have to be supplied by you.
Other pool rules apply such as no roughhouseing, no diving if it is not deep enough, no running around the edge (if it's an in-ground pool), etc. Go over all of the rules with all the kids before anyone gets in the pool. Write the rules on a board and post them by the pool, too, so no one can "forget". Rule breakers will have to sit out for an established period of time (5 or 10 minutes). Believe me, if it's hot out and they see their friends having fun, they won't do it again.If they do, they have to immediately go home.
I know this may sound really harsh and restrictive but just think how you'd feel if something happened to one of your kids or one of their friends. Establish all the rules up front, enforce them with no exceptions and they'll get it pretty quickly and have a good time.
Get yourself a baby sling, a comfy chair with a big umbrella, and a cooler full of iced water and enjoy your new baby and your summer. Good luck!