L., I am in the camp of no swaddling. I've had eight children (yep, been pregnant eight times) and am a brain development specialist as a vocation.
Babies NEED to move in their sleep. We are built to move. When you were pregnant, you would have been scared spitless if the baby had stopped moving, yet once they are born, we wrap them up so tight that they CANNOT move. I think it makes no sense. In babies, movement equals health.
Of course there will be an adjustment time for your son if you transition away from swaddling. It may be difficult for you, but it is growing his brain and helping him organize his pons (specific area of the brain). This is a long-term investment worth any hassle of the transition.
PLEASE consider letting him sleep the way he was built to.