Starting a podcast is not too hard, but it can be hard depending on what you talk about. To edit and record audio, I recommend using Audacity. You can also use it for audio editing. If course, you will need intro music too. I bought mine on fiverr for an easy 10 bucks, and it the person I bought it from let me choose from a large selection. It is pretty easy to get the hang of audacity, and if you can't, you can always search Google. Probably the thing you want most for your podcast, though, is a really good microphone. I bought a 20 dollar one on Amazon, but later had to find out I needed a USB adapter, too (it cost $8.00). That was actually the one that I used for my YouTube account, but after the first two recordings with my podcast, the USB adapter broke. I ordered a really good podcasting microphone yesterday for $78, and I chose one day delivery. I expect it today, and might record and episode to be put up later. Of course, after you buy all of your equipment, you will need to upload your recordings. I made my preview podcast episode on SoundCloud, but now I use Podomatic. It is really simple!
Also, I would write down the podcast before recording, so you know exactly what to talk about. Eventually, when I am able to talk a lot better, I will probably just have the topics out for me to remember.