Recent News and Articles

« Back

Opus, AAC, Vorbis, MP3 at 96 kbit/s

Opus, AAC, Vorbis, MP3 at 96 kbit/s

This listening test follows the similar @64 one, finished not so far ago. Codec contenders are the same with a few exceptions:

  • Opus version was updated from 1.0.1 to 1.0.2
  • Lame mp3 encoder added (3.99.5)
  • QuikTime encoder was updated but without any quality improvements; in this test it is used in True VBR mode.

Final list of contenders:

AAC VBR@89.8 (Winamp 5.63) - VBR, AAC LC

  • AAC Encoder v1.04 (Fraunhofer IIS) from Winamp 5.63: Variable Bitrate, Preset 3
  • The encoder is also available as CLI utility and can be downloaded from https://github.com/tmkk/fhgaacenc

AAC VBR@92.1 (QTime 7.7.3) - TrueVBR, AAC LC

  • QuickTime (7.7.3) AAC Encoder via qaac 2.18 (CoreAudioToolbox 7.9.8.2): qaac -V45 ref.wav
  • The encoder is included in such Apple products as iTunes, QuickTime, Safari; though not all of its features can be reached through these applications. For greater flexibility the CLI utility qaac can be used
  • download from https://sites.google.com/site/qaacpage/

AAC VBR@90.0 (NeroRef 1540) - CVBR, AAC LC

Vorbis VBR@90.4 (Xiph 1.3.3)
OggEnc v2.87 (libVorbis 1.3.3): oggenc2 -q2.2 ref.wav

Opus VBR@90.7 (libopus 1.0.2)

  • opusenc --bitrate 90 ref48.wav (44.1/16 -> 48/24 by Audition CS6)
  • The encoder operates at 48kHz only, so for other sample rates (as in our case) resampling is required. Some good quality resampler is already integrated into the codec but we used the external one from Audition CS6; the complete test files production chain: reference test files 44.1/16 -->> 48/24(Audition CS6) -->> opusenc -->> decode by foobar2000(48/24) -->> 44.1/32(Audition CS6) -->> test files production
  • download from http://opus-codec.org/downloads/

mp3 VBR@90.2 (Lame 3.99.5) - MPEG-1 Layer 3, VBR

  • encode: lame -V6.9 ref.wav
  • For this quality parameter (-V6.9) Lame down-samples the input from 44.1 kHz to 32 kHz. Decoder's output was up-sampled back to 44.1 with Audition CS6; the complete test files production chain: reference test files 44.1/16 -->> Lame -->> decode by MAD(32/24) -->> 44.1/32(Audition CS6) -->> test files production
  • download from http://www.rarewares.org/mp3-lame-bundle.php

In addition to usual live ratings on 96kbit/s page there will be detailed results of this test when corresponding ratings reach 1% accuracy (example of the detailed report is in the previous 64 kbit/s listening test).

***

SoundExpert reminds you that final conclusion about audio quality of any device added can be made only after finishing corresponding listening test. So we encourage you to take part in testing. SE ratings exist only thanks to visitors like you. Testing is short and easy, visit our Testing Room for simple 1-2-3 instruction.

Please, remember, downloading test files from SoundExpert you will not necessarily get the ones of the newly added codecs. SoundExpert testing is blind testing – you don't know the device you test. This is the only reliable way of getting true audio quality ratings, free from any cheating and various human senses other than perception of sound quality. After you have sent your grade you will see the device you tested.

Best regards,
SE Team.


Results of the test

In addition to usual results on 96kbit/s page here is a detailed analysis of submitted grades for these codecs.

SoundExpert listening test of Opus, AAC, Vorbis, MP 3 @96 kbit/s (11.2013)


All grades submitted for the codecs are below. Numbers in brackets indicate that those grades were discarded by SE automated post-screening mechanism (Median +/- 1.5) and were not used for further calculations. Reference sound samples (BAH, BAS, CST, ...) can be found here.

 

                     AAC VBR@89.8 (Winamp 5.63)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |                                       [1]   [1]       
*7* |                           [2]                         
*6* |   [2]   [4]   [2]   [1]   [3]         [2]         [1]
----+-------------------------------------------------------
*5* |    11     6     8     4     9     4     3     5     4
*4* |     3     6     9     9     4     8     9     8     6
*3* |           1               [1]     1     2     2       
*2* |                                                       
*1* |                                                       
----+-------------------------------------------------------
NN  |    14    13    17    13    13    13    14    15    10
Mean|  4.79  4.38  4.47  4.31  4.69  4.23  4.07  4.20  4.40



                    AAC VBR@90.0 (NeroRef 1540)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |                                                       
*7* |   [1]               [1]                               
*6* |   [1]   [3]   [1]         [2]   [1]   [2]   [2]   [1]
----+-------------------------------------------------------
*5* |     8     3    15     5    10   [1]     6     4     3
*4* |     6     9     3     9     3     2    10     8     6
*3* |   [1]                     [1]    11                 3
*2* |                           [1]                         
*1* |                                                       
----+-------------------------------------------------------
NN  |    14    12    18    14    13    13    16    12    12
Mean|  4.57  4.25  4.83  4.36  4.77  3.15  4.38  4.33  4.00



                    AAC VBR@92.1 (QTime 7.7.3)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |                                                       
*7* |                                 [1]   [2]             
*6* |   [2]   [5]         [1]         [2]         [1]   [5]
----+-------------------------------------------------------
*5* |    13     4           1   [1]     1     2     6    10
*4* |     8     9     7    11     6     8    11     8     8
*3* |   [1]     1     5     4    12     3     3     1   [1]
*2* |               [1]           1                         
*1* |                                                       
----+-------------------------------------------------------
NN  |    21    14    12    16    19    12    16    15    18
Mean|  4.62  4.21  3.58  3.81  3.26  3.83  3.94  4.33  4.56



                    Vorbis VBR@90.4 (Xiph 1.3.3)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |                                       [2]             
*7* |   [1]   [1]                                           
*6* |   [3]   [2]   [1]                     [2]   [1]   [1]
----+-------------------------------------------------------
*5* |     4   [2]     2   [1]     1           4     1    11
*4* |     7     4     9     4     9     3    11     7     3
*3* |     1    10     3    10     9    11     1     7   [2]
*2* |                                   4                   
*1* |                                                       
----+-------------------------------------------------------
NN  |    12    14    14    14    19    18    16    15    14
Mean|  4.25  3.29  3.93  3.29  3.58  2.94  4.19  3.60  4.79



                    mp3 VBR@90.2 (Lame 3.99.5)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |   [1]                     [1]         [1]             
*7* |               [1]         [1]               [1]       
*6* |   [3]   [1]   [1]   [1]   [1]         [3]   [2]   [1]
----+-------------------------------------------------------
*5* |     9     2   [1]     1                 3     4     6
*4* |     5     9     8     7     2           5    14     6
*3* |           2    13     6    13     4     7     3     4
*2* |                                  11                   
*1* |                                                       
----+-------------------------------------------------------
NN  |    14    13    21    14    15    15    15    21    16
Mean|  4.64  4.00  3.38  3.64  3.13  2.27  3.73  4.05  4.13



                    Opus VBR@90.7 (libopus 1.0.2)
    |   BAH   BAS   CST   FMS   GLK   HRP   LOB   MOF   QRT
----+-------------------------------------------------------
*9* |                                                       
*8* |                                                       
*7* |                                                   [1]
*6* |         [4]   [1]   [1]                           [1]
----+-------------------------------------------------------
*5* |     4     3     2     7                 3     2     2
*4* |     7     8    14     7         [1]     6     8    13
*3* |     5     1     2           2     1     8     4     6
*2* |   [1]                       8    11     1   [1]       
*1* |                             1                     [1]
----+-------------------------------------------------------
NN  |    16    12    18    14    11    12    18    14    21
Mean|  3.94  4.17  4.00  4.50  2.09  2.08  3.61  3.86  3.81

 


Previous graph of the same results (for archival purposes)

Version 2.0 (11.2013)

  • bootstrapped confidence intervals of resulting means (black ones) are computed using nine sample means and shows therefore consistency of codec performance with different types of audio material. In case of small number of test samples (9) confidence intervals of resulting means computed this way have little to no meaning (statisticaly incorrect). In final version of the graph standard/usual method was used (using all grades collected for a codec).

SoundExpert listening test of Opus, AAC, Vorbis, MP3 @96 kbit/s (11.2013)

Full Res. PDF(16KB)SoundExpert listening test of Opus, AAC, Vorbis, MP3 @96 kbit/s (11.2013)

 

 

Comments
Trackback URL :

No comments yet. Be the first.

What is SoundExpert

SoundExpert - How it works SoundExpert (SE) is a crowdtesting service that provides audio quality ratings of sound equipment and technologies such as mp3, aac, wma ... encoders, portable players, sound cards, amplifiers ...

The ratings are based solely on results of blind listening tests when listeners don't know the particular device they test. So the values are completely unbiased and free from any marketing and advertising noise.

SoundExpert provides ratings thanks to visitors who take part in testing. The more number of participants - the more devices and technologies could be tested. The ratings are computed in real time while new grades are returned by visitors. The testing procedure is simple and short enough that anyone, including you, can participate. Just download a test file (<3Mb), listen it (<15s) and send back your judgment. Details are in test file. The more accurate grades - the more reliable ratings.

SoundExpert is independent non-commercial research project. It's in beta state because it uses new audio metrics and corresponding listening test design which are not widely adopted by audio engineers. The key feature of this new method is possibility to hear sound artifacts which are normally beyond threshold of human perception. So, please, use these ratings with caution. Detailed but still easy to understand explanation of SE internal mechanics is in the article "SoundExpert - How it works".

The main idea behind this service is consumer control over sound quality measurements of various audio equipment on the market. SE, being a distributed human project, combines tiny efforts of audio enthusiasts for carrying out this task.