Are you sure that he's getting NOTHING from your breast? Have you squeezed it to see if anything comes out? I ask because at one point my baby--brief period--would get the letdown and then get annoyed with having to suck for the good stuff. It threw my body off for a while. If you want to continue to nurse, then "wear the baby" as much as you can, and pump on a schedule. It's hard, but I set my calendar to pump three times a day at work. I feed him at about 6am, pump at about 9am, noon, and 3pm. Then, I am back home at 5pm. If he's not ready to eat after so long, then I just pump. When I can't get to the pumping area at work exactly when I want to, I sit at my desk and squeeze, just to give it some stimulation, so my body knows that there is a demand, even if it isnt' always the same. I have a good electric pump and a hand pump. I use that hand pump when I don't have time to set up the big one or when I just want more stimulation or to empty out after a feeding. If you have a hand pump, use it for a few minutes while you walk around doing...whatever. I use one hand for that and the other for pouring juice. It doesn't take much, and it doesn't seem like much, but it does make a difference.
Don't beat yourself up for having to supplement with formula. We do what we have to do to take care of our babies, and that all depends on what works for our families. Stay encouraged and feel empowered to do what works for you.