yeah, those covers work great. some have zippers so that you can leave the cover on and open the cover up. the thing about babies is that you should NEVER put a baby in a car seat in their snowsuit or coat. these covers give you the ability to put baby in there and protect them from the weather. you can cover a buckled baby with blankets and this cover and they will be protected from the winter wind.
i HIGHLY recommend buying one of these. these covers are one of the ONLY things that car seat clinics support or encourage people to purchase as extras to the car seats.
heres a link to one of the better designs. there are some that act like a blanket too and go under and over baby, those again would not be safe.
http://www.amazon.com/Jolly-Jumper-Sneak-Infant-Carseat/d...
and remember; even if you have had a kid before, car seat laws change, and we forget the little details. to be sure that you are buckling your child into the car properly and tightly, make sure to find and go to a car seat clinic. call your local or nearby fire department; they usually offer the classes. for example; the recommendation is to keep kids REAR facing for as long as possible. for larger, taller babies the britax brand is very good.