If breastfeeding is important to you then you should stop rice cereal. I know 4 months is an ok age to start it, but obviously she's not eating enough breast milk, so its counterproductive. She needs not only the nutrition of breast milk, but also the fluids. She's too young for adding water (wait until 6 months before doing that). Keep track of wet diapers, did they go down? Up until 1 year old the MAJORITY of baby's nutrition should be from breast milk or formula, solids should be introduced as an opportunity to experiment with new tastes and textures, not the majority of nutrition. This is why I'm worried about your situation, she's now eating LESS.
Now about the teething. Yes that's probably bothering her and might be why eating cereal is more appealing then nursing. BUT nursing is better for her. You might need to feed her lots of smaller feedings throughout the day, at least every 2 hours, to get your supply up, and also to get her to intake more. Keep track of her weight and wet/dirty diapers, this will let you know if she's getting ENOUGH. Every baby is different, so you just want to see that she's growing, putting out enough and content.
Using a pacifier to get her to calm down after feeding is worrisome too. Kids can pacify themselves instead of eating (my first did this and ended up in a weight crisis), they don't always know how to self regulate. Around 3-4 months old kids can start getting distracted from breastfeeding, so you have to keep working with them. I think it took about a month for my second to get through this stage and stop getting on off on off on off throughout the feeding. Try to be sure the nursing environment is as distraction free as possible. Also try swaddling during the feeding, I had to do this with my second when she was about 6 weeks old, did it until about 3.5 months when she outgrew the swaddling blanket. She's go WILD and pinch/scratch me and flail about. The blanket got her to focus. She didn't LOVE it at first, but it did take away her ability to go crazy and made it easier to keep her on the breast and eating.
One last thing. As babies age they can eat faster, so how are you measuring that she's getting less, the time it takes for her to eat, your feeling of fullness, amount pumped? Breasts don't always feel engorged when they're full of milk, however if they're loose/floppy then your supply is low. Pumping isn't a great way to measure because babies can get more then you can pump, however if you're comparing what you used to be able to pump compared to what you pump now, then you can see if its more or less. Try to answer these questions in your mind to be sure she's getting enough. Also be sure you're getting plenty of fluids (at least 1/2 gallon) per day, eating balanced meals and getting sleep. What about your stress level? If your production is low that could cause her to be frustrated as well.
Breastfeeding is complicated and affected on many levels. I've learned a lot since my first one, she had lots of issues (bad latch, inefficient sucking, weight crisis) and so did I (cracked and bleeding, mastitis, mega stress, post par anxiety...) so when she had her crisis the doctor insisted on supplementing with formula and after that she refused to nurse. It was devastating and since that experience I learned about LOTS of things that went wrong all along the way. The second experience was completely different and successful and though we had some problems too it helped having knowledge and better advise to deal with them.
Best wishes!