Dad needs to handle this situation, not your sister. This girl is old enough to know that your sister is not her mother and, if her own parents won't impose limits, why should she take orders from your sister? Anything your sister says, any limits that she imposes on her soon-to-be stepdaughter will most likely backfire on her. Your sister and her fiance need to sit down and talk about the situation, talk about reasonable rules and boundaries for his daughter, he then needs to talk to his daughter about what had been discussed and then back up your sister up 100% from there on out. There has to be a united front.
The only other thing your sister may want to try is start talking to this girl about careers and colleges -- really selling the idea of going away to a great school, getting a great career and living a really nice lifestyle -- and explain to the girl that in order for her to do all this, she needs to study really hard and get enough sleep so that she can do well in school and hopefully get a scholarship. If the girl is not motivated or is not a dreamer, then this probably won't have influence on her.
Wishing your sister and her new family the best of luck.