I have had three kids who were speech delayed, 1 typically developing speech kid and two infants. What you are talking about is definitely something you need to follow up on, and IMO is cause for concern, but not for freaking out (I am not saying you are freaking out). What I would suggest is that you get your child evaluated by a speech therapist who understands apraxia (since that is what you think his difficulty relates to). My son had apraxia along with autism, we found a good evaluator at Nelson Therapy, I don't know if she is still there and can't remember her name - but she was a supervisor to our speech therapist and they did a double visit.
From what I know about apraxia, doesn't exactly match up with what you are saying about your son, but you know him better. Regardless it does sound like he is having delay issues (don't buy into the "boys talk later" thing. If someone tells you that, ask them how do you tell the difference between a boy that is delayed vs one who is late? There is no answer to that, and waiting makes things more difficult on him and your family.)
You could also have him evaluated by AzEIP ###-###-####, which is a state run Federal program for children ages birth to three (it runs parallel to Special Education in the 3-21 program found in public schools). It is free for an evaluation and services, however your child has to be 50% delayed (or have multiple areas of delay totaling 50% or an established condition like Cerebral Palsy, autism, etc).
If you have the insurance, I would definitely have him evalutated by someone who is familiar with apraxia as well as speech delay. MAke sure you get a copy of the report. If they are contracted with AzEIP or DDD then they should be able to tell you if you would be eligible for services through them, but you would then need to go through that application process (but give them the copy of the report you already had, so you don't have to get your child re-evaluated, and it should take less time that way).
If you want to contact me for more information, I can help you out more.