If your son wants hermit crabs, I'm not sure he'll be happy with a Goldfish, at least at first. We had hermit crabs when the kids were in elementary school. They were pretty boring, and the kids lost interest in them fairly quickly. Guess who ended up taking care of them?
If you decide to go the fish route, Goldfish can be amazingly hardy. In fact, aquarium enthusiasts will usually buy them as starters for a new tank because they help to set up the biotic environment for more delicate freshwater tropicals. I don't usually keep them past the set up stage, but right now I have two that I inherited from a friend who was moving away, that I've actually grown quite fond of. They'll come to the front of the tank when I pass by and "beg", and will allow themselves to be hand fed (like, right from my fingertips or off of a spoon).
As with any fish, the trick is to make sure the tank is the right size for the fish and has a good filtration and temperature system. People try to keep Goldfish in little bowls, which do not provide enough oxygen exchange for the fish to survive. The rule of thumb is that you need 1 gal water/ small fish. Right now, mine are housed in my 30 gal tank with some large Gouramis, a 9" Peacock Eel, and a few small community fish, including a bushy-nosed Pleco who keeps down the algae and a couple of different types of Loaches who keep the snail population down. They're definitely holding they're own, and because of the size of the tank they're in, have grown to be quite large (about 4" long, each). They're about 3 years old and originally were won at a carnival, showing that these fish can survive very well if you take proper care of them.
Most Goldfish are cold-water fish, so you don't need a heater unless your house gets below 60 degrees. You should get a good filter, though. The best best for a good filter is an over the side of the tank type with a magnetic impeller that drives the water through 3 types of filtration (particle, carbon, and biotic). You can get a decent one for a 10 - 30 gal tank for about $30 at PetSmart or PetCo. Other than that, it's worth keeping a couple of live plants in the tank planted in gravel or aquarium sand. If you don't want to deal with snails, make sure you buy plants that are guaranteed to be snail clean (they usually come packed in plastic tubes.) The filter will keep the tank clean for several months before you really have to clean it and the tank, although it's better to do a 25% water change of the tank every couple of weeks, at a minimum.
It is important to NOT overfeed the fish. You can use flakes, which are fine, but only feed a small amount at a time, so that you don't get a lot of decaying food at the bottom of the tank. Feed only as much as you notice the fish will eat in 5 minutes, and you really only need to feed every other day. I feed mine frozen (defrosted) bloodworms, mainly because there are other fish in the tank that need this type of food and the Goldfish gobble it up, too. In any case, I figure that wild Goldfish don't eat flakes. They eat worms, algae, and small bugs, so it's better to feed them what they would eat naturally.
Keep in mind that Goldfish are schooling fish, and are not particularly territorial. They're actually fairly social, and do better when there are at least two of them in a tank. Mine hang together or swim with the Gouramis, but take off if the Gouramis start picking at each other.
Starting with Goldfish is a great way to develop an interest in aquatic life, and can serve as an introduction to keeping tropical fish as a hobby. I love my aquaria (I have four, ranging from 10 - 65 gal). Watching the fish is very calming a soothing, and observing their behavior can be very interesting.