I'd like to start off with saying that I am a mom to a 5 year old and a two year old. We also have 3 large dogs (60lbs, 70lbs and 90lbs) and 4 always-indoor-only rescue cats.
We had the animals before we had the kids, and just expected "everyone" to adjust when we had kids. My daughters would sometimes chase them, but we always had cat towers and gated off rooms where the animals could be safe. We never once had a bite or a scratch, and it's been over 5 years.
As far as your cats go, it is a very real possibility that the other cats in your neighborhood are terrorizing at least one of them. They all had their turfs, and they were suddenly required to carve out a piece of someone elses turf for his/their own. In addition, he/they no longer have the loving comfort of snuggling up to you.
In addition to the dangers from cars, dogs, other cats and possible poisons, there are so many diseases they could get! Cat bites from cat fights almost always cause abcesses as well.
I strongly urge you to let them at least be an indoor / outdoor cats again (with emphasis on indoor and outdoor only once in a while.) Snuggle them up after baby is in bed and give the skinny one lots of kitty treats / high calorie food. I'll bet you'll be amazed at how fast he regains his weight.
K.
PS If you are feeding him outdoors, he may not even be getting to eat his food. Other cats may be chasing him away from it.