S.R.
I have a touch of this on the backs of my arms and now on my legs after my 3rd pregnancy. My daughter has it as well but much worse. I feel horrible for 'giving' it to her!
I have used Alpha Hydrox lotion on my arms and while it doesn't happen overnight, my bumps on my arms do actually go away if I use it consistently (every morning). But like I said, mine is not super bad on my arms. I use the lotion on my daughter now, who is 2.5 yo. I was concerned using it before now, it would sting but she has no complaints. It does reduce her bumps but not as significantly as it does for me. :(
I find it at Ulta for about $10 a bottle. Only a little is needed so it lasts me a while. Here's the website so you know what you are looking for. Good luck!!
http://www.neotericcosmetics.com/product-details.aspx?iid=67
(I realize this is a facial lotion but it works on my arms!)