It sounds to me like you need to visit Littleton before you make any major decisions. Littleton is not a big city. Gather lots of information before you make a major decision.
You can find good people wherever you go. Just because a city is a bit larger and out of your comfort zone does not mean you should forgo an opportunity.
Think of the potential GOOD opportunities for your children as far as education, more opportunities for work, volunteering, etc. College opportunities.
I was raised in a very rural town of less than 3000 and very sheltered. My goal was to get out of that state and town as fast as I could because I hated being so sheltered in a place that was not as pure as everyone believed. There were issues in that town and a lot of it was swept under the rug but they made big news when it came out that some church leaders (not Catholic) were pedophiles and had been abusing boys and girls for years.
SO, there are bad and good people everywhere. I would not restrict myself from an opportunity out of fear.
On another point, you mention bonuses.. Keep in mind that a bonus is not a given. DO NOT count any bonus in any of the financials as you consider this move. Companies cut and re-evaluate bonuses often. A lot of people will take a job based on a bonus and end up in financial trouble.
If you look at the financial situation without the bonus, you can also see how you manage 5 college educations and your retirement plans. You have a lot to think about that is not just money related.
I don't see this as a test from God and I just can't understand why you are so scared about this decision that you didn't enjoy your holiday. It sounds to me that you are very set in your ways and just don't want any change but change can be good. You may get to Littleton and realize how much you have been missing because of fear to make change.
I hope you do not instill or project your fears to your children. Your children deserve a chance to grow and experience without living in fear.