D.N.
You can definitely try to switch cold turkey because some kids don't mind.
I have two boys (5 and 2) where one switched cold turkey (yippee!) and the other one refused (dang!).
So in his case we had to give him straight formula until he trusted it again and then we started diluting the formula with milk until it was all milk. Every few days we increased the milk part. The key is to add just a little bit so that he can barely taste the milk part else he will refuse this trick.
This worked well for us and we used this trick later on to wean them off their bedtime milk to switch them to water as well.