Opus, AAC, Vorbis, MP3 at 96 kbit/s

Published Date 4/27/13 11:18 PM

This listening test follows the similar @64 one, finished not so far ago. Codec contenders are the same with a few exceptions:

Opus version was updated from 1.0.1 to 1.0.2
Lame mp3 encoder added (3.99.5)
QuikTime encoder was updated but without any quality improvements; in this test it is used in True VBR mode.

Final list of contenders:

AAC VBR@89.8 (Winamp 5.63) - VBR, AAC LC

AAC Encoder v1.04 (Fraunhofer IIS) from Winamp 5.63: Variable Bitrate, Preset 3
The encoder is also available as CLI utility and can be downloaded from https://github.com/tmkk/fhgaacenc

AAC VBR@92.1 (QTime 7.7.3) - TrueVBR, AAC LC

QuickTime (7.7.3) AAC Encoder via qaac 2.18 (CoreAudioToolbox 7.9.8.2): qaac -V45 ref.wav
The encoder is included in such Apple products as iTunes, QuickTime, Safari; though not all of its features can be reached through these applications. For greater flexibility the CLI utility qaac can be used
download from https://sites.google.com/site/qaacpage/

AAC VBR@90.0 (NeroRef 1540) - CVBR, AAC LC

Nero AAC Encoder 1.5.4.0 (build 2010-02-18): neroAacEnc.exe -q 0.34 -if ref.wav -of out.mp4
download from http://www.nero.com/eng/downloads-nerodigital-nero-aac-codec.php

Vorbis VBR@90.4 (Xiph 1.3.3)
OggEnc v2.87 (libVorbis 1.3.3): oggenc2 -q2.2 ref.wav

download from http://vorbis.com/

Opus VBR@90.7 (libopus 1.0.2)

opusenc --bitrate 90 ref48.wav (44.1/16 -> 48/24 by Audition CS6)
The encoder operates at 48kHz only, so for other sample rates (as in our case) resampling is required. Some good quality resampler is already integrated into the codec but we used the external one from Audition CS6; the complete test files production chain: reference test files 44.1/16 -->> 48/24(Audition CS6) -->> opusenc -->> decode by foobar2000(48/24) -->> 44.1/32(Audition CS6) -->> test files production
download from http://opus-codec.org/downloads/

mp3 VBR@90.2 (Lame 3.99.5) - MPEG-1 Layer 3, VBR

encode: lame -V6.9 ref.wav
For this quality parameter (-V6.9) Lame down-samples the input from 44.1 kHz to 32 kHz. Decoder's output was up-sampled back to 44.1 with Audition CS6; the complete test files production chain: reference test files 44.1/16 -->> Lame -->> decode by MAD(32/24) -->> 44.1/32(Audition CS6) -->> test files production
download from http://www.rarewares.org/mp3-lame-bundle.php

In addition to usual live ratings on 96kbit/s page there will be detailed results of this test when corresponding ratings reach 1% accuracy (example of the detailed report is in the previous 64 kbit/s listening test).

***

SoundExpert reminds you that final conclusion about audio quality of any device added can be made only after finishing corresponding listening test. So we encourage you to take part in testing. SE ratings exist only thanks to visitors like you. Testing is short and easy, visit our Testing Room for simple 1-2-3 instruction.

Please, remember, downloading test files from SoundExpert you will not necessarily get the ones of the newly added codecs. SoundExpert testing is blind testing – you don't know the device you test. This is the only reliable way of getting true audio quality ratings, free from any cheating and various human senses other than perception of sound quality. After you have sent your grade you will see the device you tested.

Best regards,
SE Team.

Results of the test

In addition to usual results on 96kbit/s page here is a detailed analysis of submitted grades for these codecs.

SoundExpert listening test of Opus, AAC, Vorbis, MP 3 @96 kbit/s (11.2013)

All grades submitted for the codecs are below. Numbers in brackets indicate that those grades were discarded by SE automated post-screening mechanism (Median +/- 1.5) and were not used for further calculations. Reference sound samples (BAH, BAS, CST, ...) can be found here.

AAC VBR@89.8 (Winamp 5.63) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | [1] [1] *7* | [2] *6* | [2] [4] [2] [1] [3] [2] [1] ----+------------------------------------------------------- *5* | 11 6 8 4 9 4 3 5 4 *4* | 3 6 9 9 4 8 9 8 6 *3* | 1 [1] 1 2 2 *2* | *1* | ----+------------------------------------------------------- NN | 14 13 17 13 13 13 14 15 10 Mean| 4.79 4.38 4.47 4.31 4.69 4.23 4.07 4.20 4.40 AAC VBR@90.0 (NeroRef 1540) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | *7* | [1] [1] *6* | [1] [3] [1] [2] [1] [2] [2] [1] ----+------------------------------------------------------- *5* | 8 3 15 5 10 [1] 6 4 3 *4* | 6 9 3 9 3 2 10 8 6 *3* | [1] [1] 11 3 *2* | [1] *1* | ----+------------------------------------------------------- NN | 14 12 18 14 13 13 16 12 12 Mean| 4.57 4.25 4.83 4.36 4.77 3.15 4.38 4.33 4.00 AAC VBR@92.1 (QTime 7.7.3) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | *7* | [1] [2] *6* | [2] [5] [1] [2] [1] [5] ----+------------------------------------------------------- *5* | 13 4 1 [1] 1 2 6 10 *4* | 8 9 7 11 6 8 11 8 8 *3* | [1] 1 5 4 12 3 3 1 [1] *2* | [1] 1 *1* | ----+------------------------------------------------------- NN | 21 14 12 16 19 12 16 15 18 Mean| 4.62 4.21 3.58 3.81 3.26 3.83 3.94 4.33 4.56 Vorbis VBR@90.4 (Xiph 1.3.3) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | [2] *7* | [1] [1] *6* | [3] [2] [1] [2] [1] [1] ----+------------------------------------------------------- *5* | 4 [2] 2 [1] 1 4 1 11 *4* | 7 4 9 4 9 3 11 7 3 *3* | 1 10 3 10 9 11 1 7 [2] *2* | 4 *1* | ----+------------------------------------------------------- NN | 12 14 14 14 19 18 16 15 14 Mean| 4.25 3.29 3.93 3.29 3.58 2.94 4.19 3.60 4.79 mp3 VBR@90.2 (Lame 3.99.5) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | [1] [1] [1] *7* | [1] [1] [1] *6* | [3] [1] [1] [1] [1] [3] [2] [1] ----+------------------------------------------------------- *5* | 9 2 [1] 1 3 4 6 *4* | 5 9 8 7 2 5 14 6 *3* | 2 13 6 13 4 7 3 4 *2* | 11 *1* | ----+------------------------------------------------------- NN | 14 13 21 14 15 15 15 21 16 Mean| 4.64 4.00 3.38 3.64 3.13 2.27 3.73 4.05 4.13 Opus VBR@90.7 (libopus 1.0.2) | BAH BAS CST FMS GLK HRP LOB MOF QRT ----+------------------------------------------------------- *9* | *8* | *7* | [1] *6* | [4] [1] [1] [1] ----+------------------------------------------------------- *5* | 4 3 2 7 3 2 2 *4* | 7 8 14 7 [1] 6 8 13 *3* | 5 1 2 2 1 8 4 6 *2* | [1] 8 11 1 [1] *1* | 1 [1] ----+------------------------------------------------------- NN | 16 12 18 14 11 12 18 14 21 Mean| 3.94 4.17 4.00 4.50 2.09 2.08 3.61 3.86 3.81

Previous graph of the same results (for archival purposes)

Version 2.0 (11.2013)

bootstrapped confidence intervals of resulting means (black ones) are computed using nine sample means and shows therefore consistency of codec performance with different types of audio material. In case of small number of test samples (9) confidence intervals of resulting means computed this way have little to no meaning (statisticaly incorrect). In final version of the graph standard/usual method was used (using all grades collected for a codec).

SoundExpert listening test of Opus, AAC, Vorbis, MP3 @96 kbit/s (11.2013)