I did what you started to do - limit it to naps and bedtime. Our son seemed to be using it more and more and, same as your child, tried talking with it and we couldn't understand him, plus I was tired of the "Where's the binky? Do we have a clean one?" etc. So we limited it to naps and bedtime, then just bedtime and then we just told him that he didn't need it anymore. He cried for about 3 nights and that was it. He was about 2 1/2 at the time.
I had another friend who slowly cut a bit of the tip off each day until there was basically nothing left and he gave it back to her. (However, I've since read that that can be a choking hazard.)
Another friend gathered them all up and had her daughter pick out a new toy and she had to use her pacifiers to "pay" for it and they gave them to the clerk at the store.
I also read about the Binky Fairy where you put the binkies in the window for the fairy to take to the new babies who need them and in the morning, the Fairy leaves a present in return for the binkies.
Good luck in whatever you do. Believe me, it's a lot easier to break a pacifier habit than a thumb-sucking habit (we're still working on that with our daughter!).