That is why due dates are really just "guess dates" and why so many babies end up being born by unnecessary c-section - doctor gets anxious that baby is 40+ weeks, mom isn't ready to induce, induction "fails" and baby is born my c-section and mom gets an abdominal scar for life.
You can either go along with what your doctor says, fire the doctor and get a new one, consider a midwife or just know that as you get to the end that you aren't as far along as your doctor thinks and that most first babies arrive around 41 weeks and 2 days if you do not get medical intervention!
Your doctor will also tell you that the "dating" will be perfect with the ultrasound but in reality that can be one week off in either direction as well.
Stop holding off, just do whatever you want...If I were you, I would use my own dates.
C.