MP3's

This might sound like a stupid question, but here goes. Is there any way to critically analyze the quality of a MP3 file, other than listening to it?

I tend to be a fussbudget when it comes to saving my audio and I have begun to make emergency backup files of all my audio files, not only on standard .WAV format, but also in MP3 (for the sake of size considerations).

It seems to me, a difficult task to find a good balance between size and quality. About every file I've ever heard at the 128 bit rate sounds inferior to me, but at higher bit rates the difference isn't so easy to detect.

Right now I'm erring on the side of caution and encoding my 'emergency' files at the 320 bit rate to get the best possible resolution; so that's what brings up the question.

Also, I've been using the L.A.M.E. encoder, which seems to do a fairly credible job. Are all MP3 encoders created equal?

I know that MP3 is a lossy format, but locating a particular artist or CD in my vast archives is not always so easy. At least with the MP3 format, I can store quite a few albums on 1 removable media, and at least gain access to a particular song or album until I can find the high quality archive.

GB

[Craig Maier]

MP3 compression utilizes the psycoacoustical principle of signal masking. That is to say that a louder signal tends to mask lower level signals at other frequencies.

To objectively and quantitatively measure the performance of one bit rate vs another would be a fairly complex measurement. I can imagine how to do it, but I never actually have. It would involve the use of multiple tones at various signal levels (created by the Make Waves generator) and looking for the point at which a particular signal completely disappears after compression. If it becomes of great interest to a wide group of folks, I will think about developing a test procedure and writing a paper on the subject.

However, a more subjective yet visual method could use comparisions of one bit rate vs another using the Spectrogram in the Forensics version. I suspect that the spectrograms will be dramatically different between the bit rates used on a given piece of music.

I think that you could download the demo version of the Forensic program and use that functionality so long as the piece that you are using for evaluation is no longer than a minute in length. But, I must say that I have not made this type of comparison either; I just think that it would be obvious on that method of audio signal display.

Maybe someone is willing to try it and report back what they learn using the Spectrogram.

The demo version of the Forensic software can be found at:

www.diamondcut.com

[Geebster]

Thanks for the info Craig.

It seems that the MP3 compression scheme doesn't always yield exact results with differing types of music. With my listening only tests, some pieces sound fine at 192 kbps bit rate while others sound just OK. And if this is the case, a person would basically have to run tests on each piece of music and inspect the spectrogram to see what bit rate would yield the best sound vs. size ratio.

I still would very much be interested in learning about what the spectrogram would tell us with regards to MP3's.

I have never even played with the forensics version, so I feel I am not especially qualified in doing this. If nobody steps up to the plate that is familiar with the forensics version, I will be more than happy to at least try it. I'll give others a bit of time before I tackle it.

GB

[Craig Maier]

A lot of non-forensics users seem to like the spectrogram for general signal anaysis. Since the demo is free and does work for 1 minute samples, I think that it would be a good tool to have in your arsenal, especially considering the price.

To get a benchmark, I would choose one short piece of music, perhaps 10 seconds worth. Pick a good file having excellent fidelity in its raw form. Then, convert it to MP3's at the various bit rates. Then, reconvert these back to .wav using Diamond Cut and then compare each of their spectrograms. If you have a color printer, you can print out each spectrogram and compare them by laying them all out on a table and observe the differences.

If you want to understand how to use the spectrogram, your DC6 users manual has information on that. Check it out. It could be an interesting investigation.

And, yes, I do agree that the type of music makes a big difference when using the MP3 format. For example, classical music brings out the worst characteristics of lossy compression.

[Geebster]

A lot of non-forensics users seem to like the spectrogram for general signal anaysis. Since the demo is free and does work for 1 minute samples, I think that it would be a good tool to have in your arsenal, especially considering the price.

Yes. Point well taken. I believe you could get a pretty good feel of what the whole file will act like with just a minute sampling anyway.

Tomorrow I'll play with it some, and see if I can gain anything productive from my experimentation.

GB

[Dan McDonald]

I did look at mp3s at various bit rates, all taken from a wav file, and using different encoding methods a few years ago, just for my own personal knowledge. As I recall, you can see the cutoff for mp3 very clearly at a particular frequency (I think either 12 or 14 kHz) for most bitrates and methods, but when you get into higher sampling rates it was more difficult to tell the difference between an mp3 and a wav. Of course, as they got higher, the file size difference became a lot less also.

In the end, I decided that I didn't want to bother with mp3 because I hated the idea of losing something.

Dan

[Geebster]

In the end, I decided that I didn't want to bother with mp3 because I hated the idea of losing something.

Dan

Thanks for your input Dan. Yeah, I hate to use MP3 as well, but sometimes it takes me a few days to locate my original files or CD's and if a student wants to work on something, I like them to hear the original song; that is if I have it in my library somewhere. And with MP3's being smaller, it makes locating them a bit easier.

GB

[DougMac]

Metadata?

I went to check to see if I could find out if this type of info is stored in the header file, but they've got us so clamped down at work that any reference to MP3 in a search is blocked.

You might want to do a search. If the rate is stored in the header, I'm sure there's program that will display it. That would take the guess work out of the determination.

Just a thought.

Doug

[Oskar Maier]

I think that the objective of the question is more along the lines of what quantitative measurements could be made to evaluate the differences in the quality of audio data as a function of the encoded bit rate.

Your header idea would allow us to know how a file was encoded. However, one still may not know what the differences in performance would be based on various bit rates.

So, it seems that the test would be to create multiple files at various bit rates of a given piece of music and then compare them after decoding them back to .wav's with the spectrum analyzer or, better yet the spectrogram.

btw - Oskar Maier is my pseudonym; we were having a problem with new members signing up on the forum, and I forgot to re-sign in under "Craig Maier".

[Geebster]

MP3 test results.

Hi Craig (and others interested in this thread)

I performed a series of tests that you recommended and it was a real eye opener. It's one thing to guess what is going on and quite another to see the proof in black and white (or in my case red and white).

I recorded a sample of Borodin's Polovtsian Dances as found on "Mad about the Classics" on Deutsche Grammophon #439-513-2 GMA.

The various files I recorded with their approximate file sizes are listed below:
1. Borodin Original .Wav file... 2286Kb
2. Borodin Mp3 128 bit rate... 209Kb
3. Borodin Mp3 192 bit rate... 313Kb
4. Borodin Mp3 320 bit rate... 522Kb

I downloaded the demo version of the forensics program and printed out the spectrogram. Naturally, the original waveform looked the best with musical information up to 20k.

The 128 bit file looked, just as Dan suggested earlier, the worst with virtually no musical information present past 15K. Not only that, it was like the file was given a "crew cut" (similar to a clipped file) with a flat top across the whole length of the file.

The 192 bit and 320 bit files were similar with very subtle differences between them both. Both had close to a upper limit of about 18K, but were not flat topped at all, like the 128 bit file. You could tell there was more overall musical information present in both of those files in comparison to the 128 bit file.

Finally, I listened to all of the said files to see if I could hear any glaring imperfections. They all sounded quite musical, with a good approximation of the original. Even the 128 bit file sounded rather good, but with a not as crisp high end as the result of no musical content above 15K.

The 192 and 320 bit files were very hard to distinguish by listening tests alone. They sounded just about as good as the original as you could get using a compressed file. The most notable difference is that the original sounded more "natural" and "open" than the 2 compressed files.

My conclusion in all of this is that I would personally only use or listen to a 128 bit MP3 file only if there was nothing else available. In my past listening tests, I always thought they sounded inferior and the above tests prove the point. For non-critical applications or in my case as emergencey files, the 192 and 320 bit files sounded pretty darn good. No replacement for an original .Wav file, but in a pinch they would do quite nicely.

If I can further explain any of this, please let me know.

GB

[Craig Maier]

Nice summation, GB. Can you tell us if the overall shape of the spectrograms differed in any other way other than the "crew cut" that you noted? Was the spectral distribution different from one rate to another down at lower frequencies?

Just curious.

ps - btw - you can change the color palette for the spectrogram (spectrograph) under the preferences menu. You can choose something that will provide more colors which may be more revealing in terms of differences in bit rates.

[Geebster]

Craig...

Yes, I found the preferences last night and found that at least on my equipment, the red and white display showed the most information about the file.

All of the MP3's looked surprisingly good at lower frequencies, with the exception of the 128 bit file that besides the "crew cut" also had a noticeable lack of information across the whole length of the file at about 11.5-12K. Information was still present there, but not at the same intensity as the original or even the higher bit rate MP3's. The 192 and 320 bit rate MP3's didn't have this problem.

As noted earlier, the biggest differences were at the higher end of the audio frequencies. The higher the bit rate, the more musical information that was present in this area.

GB

[Craig Maier]

It makes intuitive sense that the lossy compression systems would compress the higher frequencies to the greatest degrees. The higher a signals frequency content, the more data is required to represent it accurately and thus, to store it. Squelching the high frequencies wherever possible would save a lot of storage space which is the primary goal of lossy compression.

[Geebster]

It was a good experiment. I've never been a big fan of MP3 anyway, but it proved to me that there is no free lunch. You can't save that much data space in a file and NOT give up something in return.

If I do have to use the MP3 format, I now will use the highest bit rate as possible, as it more closely resembled the original file.

I'm not saying that MP3's are bad, but they are no substitute for a non-compressed file.

GB

[Craig Maier]

There are non-lossy data compression schemes that cuts the file size in half. One of their file extensions is .ape (Monkey Audio I think). Also, Windows Media Player versions 9 and 10 supports a non-lossy compression scheme. I believe that our own DC Audio Mentor product supports the second method, but I am not 100% sure. If anyone is interested, I will look into it (ie, ask Rick).