Have you been tested for celiac disease? If not, you might want to try cutting any wheat products from your diet, it is also in barley products. You might also have a dairy allergy. You could try cutting that out of your diet as well. (do one first then the other so you're sure which thing is a problem)
If you choose to cut stuff out, be sure you're reading all the labels, even for cross contamination stuff like "made in a plant that uses wheat/dairy". Don't just eat less of those things, cut them out completely and see if you have any relief.
Also look at your diet, no matter what you do. Do you eat out a lot, what about refined foods (white pasta, flour, bread, rice, stuff in a box or can). If you're eating a lot of these foods you're not getting enough fiber. A fiber supplement is helpful, but switching to whole grains and unprocessed foods will really help, especially with your family history. With intestinal cancers a plant based diet makes a big difference in prevention and also during treatment. If you choose to include meat and dairy in a plant based diet, keep it to the minimum, use as a seasoning, not as a large chunk of your meal.
Best wishes, I hope you can find some relief!