Do be honest with them and tell them you don't know where he is and you'd appreciate it if they'd stop asking you. Have you told them he is running around with other women? If not, tell them. Honesty is almost always the best policy. Be courteous and sensitive to their feelings while you express your reasons for leaving him. Tell them you want nothing more to do with him as he is now and so please stop making suggestions. Tell them they can call when he has a steady job and a home for them. Put the responsibility back on him with them.
The kids are still their grandkids, etc. It's good to let them see the kids, to keep them up to date about the kids. You can respectfully encourage them to visit. Remind the father that you are in need of a job and are looking; that you don't have the money or time for a trip, either. Let them know you know that they miss their grandkids and you'd bring them if you could but you can't.
Sympathize with them while standing firm with your decision. Once you said the above you don't have to repeat it. When they call and start in remind them that you don't want to hear it. If they persist tell them you're going to hang up and do so. It's not your responsibility to make them happy.