I had this with my second pregnancy towards the end of the eighth month and all through my third pregnancy. First it was only on my right side, and later it was on both. I wore Ted stockings to help with circulation and it helped a bit. My midwife said it was common with subsequent pregnancies, and I should see a chiropractor who specialized in pregnant and post-partum women.
During my third pregnancy I also got numbness radiating from my elbow down to my pinky finger (along the ulna) but only on the one side. I had two vertebrae out of place in my lumbar region (lower back) and one in my cervical (neck area). One appointment was all it took and I felt better than ever (though it was suggested I continue, I didn't have insurance to cover it at the time).
I would talk to your doctor, and if he/she feels it could be something besides a blood clot, maybe schedule an appointment. I can't tell you how great I felt, not just physically, but for the piece of mind. Don't be afraid to ask questions and shoot out other ideas. It's better to ask than not to know. Best wishes, and congrats on the new baby! :)