I know what you mean. Sometimes I only want the audio from the video files. There is a website called Vixy that will extract just the audio from the video into an mp3 file. Then you can download that file. All you have to do is copy the YouTube video URL into the conversion box on Vixy select what type of file you want it to be converted to (in your case .mp3), and click start. Hope it helps.

http://vixy.net