M.M.
The fever so late in the game and the persistent cough point out towards a bacterial infection (sounds like some kind of bronchitis) that developed as a result of prolonged period of respiratory unwellness of some sort (viral or bacterial).
Treat the fever above 102 and go to the doctor as soon as you can, I do not see a reason for urgent care. Regular doc will suffice.
Prolonged infections cause tissue scarring and they exhaust the immune system - so, taking antibiotics is not all that bad. Much worse would be to develop some sort of super-infection because the weakened immune system cannot fight it off and to end up on some very serious antibiotics.
I am not a fan of drugs but they are needed sometimes in the world we live where bacterias have evolved to be very resistant and our bodies cannot fight them off on their own at times.
Good luck.