Totally normal! Most of us are not completely symmetrical. Have you tried putting him on the right for a few minutes and then switching him to the left? If he's been eating for a few minutes he might not be as hungry and might be a little more patient.
It's possible you're not getting much from the left with the pump simply because the pump does not stimulate that breast very well. I had that same issue. When I pumped (and I had to pump exclusively with my youngest) I usually got twice as much from one side. I've heard that you can try different size shields and see of that makes a difference.
I don't think you have to worry about developing and infection. If you decide not to continue pumping, eventually your body will adjust and you simply won't produce as much. If you're fine with that (and don't need to have a supply on hand), you might start pumping 3 times a day instead of 4 and then after a week or so drop down to 2 pumpings a day. If you like having the supply and don't mind taking the time to do it, you could continue pumping and know that you have that on hand in case you and your husband want a date night.
Bottom line, it's really totally normal. Try not to worry about it. Sounds like your little guy is doing great!