Wow, you're very lucky to have a choice! My son is at an elementary school (k-4) where the middle school (5th, 6th, 7th & 8th)is in an attached building. He loves it! He is a very advanced reader, and now has access ( with his advanced reading group) to the larger middle school library. They also have more facilities for gym, music, arts, etc. Because the younger kids have some classes where they walk through the middle school building to the auditorium or whatever, it prepares them for the bigger school and they get to know their way around a little and get comfortable with the middle school as well as their own elementary.
His school also has a great 'mentor' program where 4th graders like my son mentor new kindergarteners and then THEY get mentored by 6th grade students. We have not experienced ANY bullying at all from older kids and I do not know of any parents in the elementary school who have. Any bullying I have ever heard about from parents has come from peers in their own age group. I think your kids will be around people of different ages for a long time and need to become accustomed to it and not be afraid of bigger kids for no real reason- don't create problems where none actually exist yet.
I have not seen any drawbacks at all from having the middle school attached to the elementary school- it has actually been a plus and my son really likes sometimes catching a glimpse of his babysitter or a friend's older brother or sister that he knows in the middle school.