Introduction: Why listening tests matter
Listening tests are how audio formats are evaluated. For example, established formats like AC-3 (from Dolby laboratories) undergo extensive tests to establish whether listeners can tell the difference between an original and compressed signal. These listening tests are conducted extremely carefully to legitimately confirm the "transparency" (similarity to the original signal) of a given format. In conducting such tests several critical test criteria must be adhered to, which Microsoft has not confirmed they have followed:
Content must be chosen which is "difficult"
When doing critical listening, samples must be chosen which are both hard to compress and likely to create artifacts which are easy to hear. Examples of such content include isolated instruments (like castanets or a harpsichord, for example), snare drums, acoustic guitars, and solo vocal passages. Many types of content are very easy to compress and sound great to a listener even at very low bitrates, such as orchestral music and loud/bright popular music (where there are many instruments and relatively quiet vocals).
Short sample lengths can mask significant objectionable characteristics
It is especially important when evaluating audio codec performance to ensure sample clip lengths are long enough to identify objectionable artifacts that would become annoying after sustained listening. This is especially critical when it comes to codecs for music content, as songs and albums can span minutes or hours, respectively of sustained listening. Short clips can be used to downplay or minimize truly objectionable artifacts, making the mildly objectionable become truly un-listenable.
What follows is a brief assessment of the performance of RealAudio G2, MP3, and the recently introduced MS Audio. These assessments subject each audio technology to evaluation across parameters that are grounded in real-world applications, and highlight capabilities that are required in order for sustained, pleasurable listening experiences.
Section One: MS Audio is not superior to MP3
Several types of audio artifacts (audible differences between the compressed signal and the original) introduced by MS Audio can easily be discerned in careful listening. These artifacts are not present in MP3, nor are they present in any of several other currently available high-quality audio format choices (such as Liquid Audio, Lucent's EPAC, A2B Music, and the MPEG-4 AAC standard). An audio format seeking to deliver a listening experience to the user which is identical to the original CD must be absolutely free of such artifacts, and must perform well enough to produce a satisfactory listening experience over minutes and hours of listening.
Note that it is much easier to hear these artifacts if you listen using headphones.
NOTE: All files are available for download only, they are not configured to stream. Right mouse click on the clip and select "save link" to download these files, left click to download and immediately play.
Pre-echo artifacts in vocals
Listen to the original uncompressed file, and then listen to the MP3 and MS Audio clips. Note that in the original signal and in the MP3 file, the voice is very clear and has no echo. Then listen to the MS Audio clips and notice the echoing, especially at the very beginning of the speaker's words.
