I havent been in your shoes as far as needing a nanny, but I know the things that would be important to me is her parenting philosophies. I want to make sure the person caring for my baby shares the same views as I do, or is at least willing to "parent" the way I would. Some examples would be not leaving my infant to cry it out. Wearing him in a wrap if he seems fussy. Not feeding him anything I dont pre-approve. How would she handle a situation that required discipline and how do I feel about her methods? Basically just making sure we see eye to eye on things that are important to me. Ask for references too!!!!
Also, from what I hear, money is a HUGE issue. When to pay, how much, how often, what if there is a cancellation on her part or mine, etc...
Oh...and if you breatsfeed, make sure she knows how to safely handle, heat and store breastmilk and how much to give. Bottle fed babies tend to be over fed by care givers (formula or breastmilk) because the flow is faster and it takes baby 13 minutes to realize they are full, and you dont want to be pumping and having it wasted like that. Breastfed babies dont need as much per feed, but more feeds per day. If you do nurse, I have a math equation to help you figure out how much your baby needs while you are away so you know how much to pump. Send me a private message if you need it.
I hope that helps. Sorry I dont have much more to offer!