S.J.
Start with a business license from your county. For Dorchester County, the office is in the main county building at 500 N Main St in Summerville. It's not off the main entrance -- you have to go around the north side of the building.
Also you'll need a retail license from the state of South Carolina and get set up to pay sales tax. Check out South Carolina Business One Stop: www.scbos.com -- great site with all the info you need. You can even file online. I just log on once a month and "file" and pay the sales tax I collected from customers.
The Chamber of Commerce also has a mentoring program for small businesses.
I'm not sure why someone below told you it was illegal. You probably wouldn't be able to open a full-out store or cafe in your home -- but just doing some baking on the side should be just fine. Basically they don't want you having customers regularly in and out of your home -- but delivery is fine.
I run a small biz out of my home in Summerville -- I do flowers for weddings (www.SweetMagnoliaFlowers.com).