You need professional assistance. Is he on an IEP at school. Individual Education Plan. If so, then you know the system and should be able to get some help from the school district. If he's acting out at home, he's also acting out at school.
Federal law requires that all school districts provide services to children with learning disorders so that they can succeed in school. My 6 yo grandson has a speech disorder as well as difficulties with behavior. A social worker came to my daughter's house and helped my grandson and his mother learn a different way of being with each other. My daughter learned some parenting skills that also help her with her daughter.
Disciplinary methods used with your other children are not working with your son. Since I don't know what you're using with them I can't give suggestions on alternative methods. I do know that with my grandson, boundaries have to be worded with just a few words and consequences have to be simple and consistently enforced. When my grandson and his sister are not getting along they both have to go to separate rooms until they are calm and can be courteous with each other. Your apd son may be the one who started the fights but his sibs are continuing them. They need to learn different ways to react to him.
The only consequence for my grandson is going to his room. This gives him an opportunity to calm down. It's like a time out but not so restrictive. He is the one who decides when he's ready to have reasonable behavior. I think this works better than time outs, taking away privileges, toys, is because the misbehavior originates in his lack of self control. Being alone in his room gives him the chance to regain control. Spankings never work because they increase the emotional and energy level. This is the opposite of what needs to happen.
I suggest that family counseling with a child therapist specializing in special needs children that includes your other children would be helpful. You son with apd is handicapped in more ways than just auditory. He needs to learn how to get along with others and is frustrated by his physical inabilities. Your other children also need to learn ways to deal with his frustration and his sense of having no control. I suggest that all of the children as well as you feel that the situation is out of control and don't know what to do to change that. A professional therapist can help you all to learn ways to get along.