A.B.
How long did you keep him off the milk? It will take a week or more to really see the full effects - and that means not just drinkng milk, but all diry products. Like another poster said, casein is in EVERYTHING!
You can try mixing rice milk and soy milk (enriched) in the meantime.
And to ditto what another poster said regarding food (prunes, etc.) we have found Pears to be very helpful with preventing constipation.
But what you are describing really sounds a lot like a food allergy or intolerance!
Good Luck!