I used to work in the 401(k) department of a large financial firm and some of your answers are actually half true because there is some confusion on terms. My shortest possible answer: "Taking" money out which so many people say is "the biggest mistake" is one thing, BORROWING money from it is a totally different thing, and depending on your situation, can absolutely make sense. There's no black and white answer to this, because there are too many factors (age, date of expected retirement, vesting date, income, employment situation, savings situation and other accounts you may have, interest you'd pay yourself vs interest you are currently paying, individual plan-specific rules, etc, etc).
I would not suggest "taking" money out because yeah, there are penalties charged by the firm if taken before retirement age AND you will be taxed on it as additional income, plus state taxes if you are charged that where you live, as well as just not having an account to build your retirement or the "free money" that your company is putting in, and money that is not in your account is not making money for you. There is such thing as "hardship withdrawals" but each plan has its own rules on that, and I would seriously not suggest that for you. To tell the truth, most plans would not accept debt payment as a hardship so you wouldn't be able to take it out without actually closing your account which would be a big mistake.
For borrowing money, under certain circumstances, it can really make sense. Yeah, you'll miss out on whatever money you'd be making if the market is doing well. If the market is stagnant, you could also miss out on losing some of that money too if you think about it. If you were to take a loan from the bank, you will pay interest to them and it will be on your credit report. This isn't always helpful (in my PERSONAL experience: we both had "blah" credit when we first got married, were working very hard to pay off all debt from our past lives, rebuild our credit reports, and save money for a house because we were in a 1 bedroom apartment with an 18 month old boy. We didn't want to pay as much or more in monthly payments for a 2 bedroom apartment as we'd pay for a house in a great neighborhood. We got all credit debt paid off, improved our credit a bit, saw that if we could come up with a certain amount down, we could get a loan from the bank for $200 more per month for a 3 bed/2 bath house with amazing energy efficiency - read lower utility bills - in a great school district and an amazing neighborhood than we were paying for the 1 bedroom apartment with NO energy efficiency. The bank was charging a substantial amount more for a loan than our 401(k) would, which in that amount of money is quite a chunk of change. We knew that as we paid off debts, we would also need to get a car -brakes were iffy and the ac was out in Tx heat, so we didn't want our credit showing another loan that would lower our standing, etc). For OUR situation, it just made sense to us to borrow the down payment from ourselves. Then instead of making monthly payments to a bank, we were paying OURSELVES back, which was easier for me to stomach than paying some bank. Instead of paying interest to the bank, we were paying ourselves a lower, doable interest rate. We took our money out at a decent time when stuff still had some value so we didn't lose too many shares by selling out at a time when the value was uncomfortably low. We could see things were going to get a little worse (they did) for the market, so it actually helped us to take some money out before the value went down, and we were paying ourselves back and helping that balance until the market improved (which it has, for many). We were responsible; I urge anyone who is going to borrow money from anything to be responsible! Yes we got the house, and 2 newer (not new, but newer) vehicles, but we are totally debt free except a modest, totally doable mortgage on the house now. We have paid our 401(k) loan back at a lower interest rate, and that helped us, it saved us money in the long run. This was years ago when we first started out, so I can't remember the exact numbers offhand, but it was something like 4.25% from 401(k) vs 8% from a bank and I know our credit cards used to be 14.99% which is ridiculous. I'd rather pay 4% than 15%! Then be diligent to pay it off quickly, which is easier to do when the interest rate is THAT much lower. It really depends on your situation. You can talk to a financial professional, call for an agent who handles your personal 401(k) plan and they will be able to look at the plan and discuss the ins and outs with you as they pertain to your specific plan. Take notes on your discussion, and use those notes to compare what would happen if you tried a couple banks OR the interest of just paying the debts with no loans. See what makes the most sense for you by looking at all 3 options on paper. You can make your decision then, based on having the facts in front of you.
As for the tax issues that people seem to be so concerned about: how about finding a high interest bank account and designating it just for taxes? Example: one bank I use has a pitiful savings account of 0.21% but the checking account is currently 4.01%, with the stipulation that we use the card 15 times every month to earn that interest. Well, we put the money that we designate to pay our taxes into that account every quarter into that account. We do use the card but it is for purchases UP TO $10 each (like running to the store quickly for milk, or if I am getting a prescription with a $10 copay, that kind of thing). It is liquid, it is making money, and it is separate because frankly we won't be using that money because we're saving it for specific needs, and we won't NEED the money but once a year (A. of every year). We make 4.01% interest on the property taxes instead of overpaying and letting the government make that money instead, or scrambling to cough up some money suddenly when it's due. This is also where the rent checks from a property we own go, to make some decent interest and be separate from living expenses so we have it on hand if something needed to be prepared or whatever. Ramsey does have good tools to help you make a budget that you can live with that includes putting aside money for things like paying your loan back, or setting aside taxes, or whatever. This will only work if all involved (husband AND wife) can be responsible though. Together, we check our numbers with the online bank statement twice a month on Sundays to make sure we're still on track, not to get onto each other but because we're a team and it takes teamwork to have it all run smoothly. I do not like these cut and dry answers where they give you a "YES" or "NO" and don't know anything about your personal situation, or even the plan rules. But you MUST be responsible and not get yourselves into trouble again. This is not an easy fix. We took our loan out to assist us with a home, at the beginning of our marriage, and we were both gainfully employed. We are not in debt anymore (except the modest mortgage), and I stress that this is important. There are no easy fixes.