My son is in the ESE program (early speech therapy) in the Palm Beach County school system, and takes the bus twice a week to what will be his elementary school in a couple of years. He just turned 3 and I had the same worries that you do.
Believe the other parents, that it is more trying for you than it is for him.
I'm not positive if it is the same for Kindergarten, but for the Pre-K and ESE kids, I was told that the bus paperwork had to be turned in by the teacher after the child had actually been to school for the first day. This meant that I had to bring him to school for the first few days (the first day, for the paperwork to be turned in, and then for a couple of days afterwards until the transportation folks get his paperwork on the bus itself). A mistake and a brave little boy did a lot to ease my apprehension, and you may want to mimick what happened just to watch things at work.
On the 1st day of school, I went to my son's daycare to drive him to "the big school". He has a friend, Lilia, who is 4 and started the program last year, so we waited with her and watched her get on the bus, then followed it to the elementary school, where we waited out front. The teacher came out and collected the kids from the bus and brought them in to the class; I gave her the paperwork went to my office for the afternoon; then came back for the pickup. After school, the teacher marched the kids out, I collected Mattie, we watched Lilia get loaded onto the bus and then we followed the bus back to daycare (it's only a few blocks from the school, so is the last stop in the morning and the first on the trip home which makes things easier).
Anyway... the 2nd day, I showed up to bring Mattie to "big school" only to find out that he had already left. Apparently, the bus had just come; they were loading up Lilia, and my son had come running out of daycare with his backpack crying "Wait for me!". The bus driver, who we had met the previous day when walking him through the process, allowed him to get on the bus without his paperwork, knowing that it was in-process! So I went to the school (it was still a half hour before class started) and I got to watch as they waited for the teacher to come out (she was a bit shocked that they had taken him, and asked me to make sure that I was there to pick him up, since the afternoon bus-driver wasn't nearly so lax) and the kids were marched single file into the school.
After school, I pulled up, and Mattie's teacher was out front with all of the kids, but Mattie and his friend, Lilia were missing. On a whim, I used my cell to call his daycare and yes, the bus was there and Mattie and Lilia were just being brought in.
The next day I managed to be early for the bus both ways, but repeated, more-or-less, the preivious day's routine, following the bus to school and back to daycare in the afternoon, watching as the children were transferred from daycare to bus to teacher and back.
Never were they left alone. The system seems to be very fine tuned (aside from letting Mattie ride that first day with paperwork "in the pipe"). Mattie's now been going since school started for the year and loves the Pre-K class. It's almost heartbreaking. There wasn't even any separation anxiety :o(
Anyway... to sum up. I'd highly recommend repeating what happened to me by accident; putting him on the bus, following it to school and watching the process at work, and repeating it for the trip home. It did loads to ease my apprehension (and it's fun watching the little trooper going through the routine).
Good Luck!