When my daughter was tiny we went through the same thing. I ended up cutting all tomato based products out of my diet after I found blood in her stool and took her to the ER. She had anal fissures because of the amount of pooping she'd done - 5 times in one day! It was awful, I felt absolutely terrible. After I cut the tomatoes out she got much better. Later on down the line I started adding them back in slowly (around the time she was about 8 months old) and she does fine with them now.
To answer your questions:
It's not likely he has a milk allergy unless you see blood or foam in his stools. To quote kellymom.com:
"A small percentage of breastfeeding mothers notice an obvious difference in their baby's behavior and/or health when they eat certain foods. Cow’s milk products are the most common problem foods and the only foods conclusively linked by research to fussiness/gassiness in babies, but some babies do react to other foods. Food sensitivities in breastfed babies are not nearly as common as many breastfeeding mothers have been led to think, however.
If a breastfed baby is sensitive to a particular food, then he may be fussy after feedings, cry inconsolably for long periods, or sleep little and wake suddenly with obvious discomfort. There may be a family history of allergies. Other signs of a food allergy may include: rash, hives, eczema, sore bottom, dry skin; wheezing or asthma; congestion or cold-like symptoms; red, itchy eyes; ear infections; irritability, fussiness, colic; intestinal upsets, vomiting, constipation and/or diarrhea, or green stools with mucus or blood."
http://www.kellymom.com/babyconcerns/food-sensitivity.htm...
As for the red diaper area, I would suggest changing the brand you use (when my daughter was that age she'd get a rash from anything other than Pampers Swaddlers Sensitive or her cloth diapers). From the description you gave it doesn't sound like your baby is sensitive to dairy. Even if he is, it doesn't seem like it'd be enough to really affect the both of you too much. I cut out most dairy from my diet when my daughter was about 9-10 months old and it took me about a week but I don't miss it! I still eat cheese on occasion but I don't drink milk anymore, and I used to LOVE milk. It's just a matter of getting used to it.
Good luck!