What is SoundExpert

Audio metric that makes a differenceSoundExpert (SE) is an independent non-commercial audio research project, which provides audio quality ratings of sound equipment and technologies such as mp3, aac, wma ... encoders and portable players (to be extended). It is the home for innovative music-based audio metric which works both for objective audio measurements and for listening tests. SE is in beta state because the new audio metric and corresponding listening test design are not widely adopted by audio engineers. So, please, use our ratings with caution.

The new listener-centric audio metric allows to perform objective audio measurements using real-life music material and various technical signals, not only sine waves. Such measurements:
• correlate well to human perception of sound quality
• are valid for digital audio
• do not contradict traditional audio metric but extend it
• are understandable to people without technical background
• can be performed at home using any recording device

The first practical application of the new measurement method can be found on the Portable players page.

The new listening test design helps to hear sound artifacts which are normally beyond threshold of human perception. The core of the new design is sound artifacts amplification technology (SARTAMP), which is backed by the new audio metric. Such amplification is particularly important for testing high-bitrate encoders and high quality audio equipment. This technology is used in SE listening tests since 2005. The tests are blind - participants don't know the particular device they assess. Resulting quality scores are completely unbiased and free from marketing and advertising noise.

Live Listening Tests @SoundExpertThe ratings of encoders are computed in real time while participants of the listening tests send their grades to the rating system. The more participants - the more devices and technologies can be tested. The testing procedure is simple and short enough that anyone, including you, can participate. Just download a test file (<3Mb), listen it (<15s) and send back your grade. Details are in the test file. The more accurate grades - the more reliable ratings.

Detailed but still easy to understand overview of SE distributed listening tests is in the paper Live Listening Tests @SoundExpert.

The main idea behind SoundExpert project is consumer control over sound quality of various audio equipment and technologies on the market. Production side deserves clearly defined feedback from consumers.

 

 

Articles

« Back

Opus at high bitrates (128-192-256 kbit/s)

Opus at high bitrates (128-192-256 kbit/s)

It looks like Opus audio format becomes popular. During last month SE received several requests to add it to the rating system. Low bitrate settings (--bitrate 59-90-96) were in the system since the first public release of the codec. Now high bitrates (--bitrate 128-192-256) are added as well.

In contrast to other VBR encoders Opus uses “non-standard” bit allocation strategy for different audio material. As a consequence encoding of less saturated music (with fewer instruments) can result in higher output bitrates than encoding of pop music for example. The table below shows that. Two different music fragments – pop-rock and classic – were encoded to Opus and mp3 formats with approximately equal VBR settings. While mp3 encoder shows usual behavior allocating more bits for more complex pop-rock material, Opus behaves exactly the opposite.

 

  mp3 (kbit/s) Opus (kbit/s)
-V5 -V2 -V0 --bitrate 128 --bitrate 192 --bitrate 256
Pop-Rock 126.6 187.3 260.6 126.5 190.4 255.1
Classic 106.9 176.3 231.2 151.7 216.8 278.1

Table 1. Resulting bitrates of encoding pop-rock music (Midnight Oil - (1996) Breathe, tracks 7-13, 26:53) and classic music (Piazzolla and Kronos Quartett - (1991) Five Tango Sensations, 26:50) with mp3 (Lame 3.99.5) and Opus (libopus 1.1) encoders at 128-192-256 kbit/s VBR settings. Opus encoder ends up with higher bitrates for less complex music.

 

Nine SE testing samples are usually encoded with lower bitrates by all modern VBR encoders because most samples are pure voices or instruments. Opus is the only encoder which produced actual bitrates higher than the set ones: 152.3kbit/s for 128kbit/s, 222.5kbit/s for 192kbit/s and 290.1kbit/s for 256kbit/s. So the ratings of Opus were placed on 160kbit/s, 224kbit/s and 320+kbit/s pages respectively. This is less convenient but more correct as all SE ratings are grouped according to actual bitrates.

The encoder operates at 48kHz only, so for other sample rates (as in our case) resampling is required. Some good quality resampler is already integrated into the codec but we used the external one from Audition CS6. The complete test files production chain is as follows:

reference test files 44.1/16 -->> 48/24(Audition CS6) -->> opusenc -->> decode by foobar2000(48/32) -->> 44.1/32(Audition CS6) -->> test files production

Opus encoder/decoder can be downloaded from http://opus-codec.org/downloads/
 

***

SoundExpert reminds you that final conclusion about audio quality of any device added can be made only after finishing corresponding listening test. So we encourage you to take part in testing. SE ratings exist only thanks to visitors like you. Testing is short and easy, visit our Testing Room for simple 1-2-3 instruction.

Please, remember, downloading test files from SoundExpert you will not necessarily get the ones of the newly added codecs. SoundExpert testing is blind testing – you don't know the device you test. This is the only reliable way of getting true audio quality ratings, free from any cheating and various human senses other than perception of sound quality. After you have sent your grade you will see the device you tested.

Best regards,
SE Team.

Comments
Trackback URL :

Jason
The explanation given for the bitrate boost on simple tonal sounds says that Opus has low frequency resolution and thus needs more bitrate.

I find that the bitrate roughly corresponds with my hearing. With simple sounds it is possible to hear the smoothness of the noise floor, robotic breath, fluttering consonants. That is usually not possible with "complex" rock music, where the noise floor is masked and where I find most simple codecs (adpcm) to sound good. Perhaps a highly accurate pair of speakers is needed to hear flaws in the stereo image in rock.

"French Male Speech", for which Opus boosts the bitrate by 30%, is the sample where artifact are the easiest to detect without training or reference. Also Bass "Matona mia cara" is easy. I can also hear the guitar part from "Castanets" get rounded and wobbly, but I can't usually hear a problem with the percussion itself.

I believe that speech, apart from joint stereo gain when it is actually mono, is a difficult signal, and sounds ugly in audio book bitrates.

I've not been very impressed with Opus on Sound Expert so far. I can hear it failing on "Harpsichord" at 290 kbit post SARTAMP. And it also fails with a low frequency pop/discontinuities even if other artifacts aren't present. I have my doubts about the validity of the artifact amplification process. The "Df" is shown to be rather low, about -14 dB. However, I don't think I will encode high bitrate Opus to be on the safe side.
Posted on 4/3/17 3:02 AM.
Serge Smirnoff
Thanks Jason for your clarification of Opus bitrates.

Talking about Opus performance at high bitrates ... I'm also a bit surprised. Probably it has "sweet spot" at lower bitrates. I'm not sure 100% in my SARTAMP process either, the research continues. But I have to note that after SARTAMP all codecs fail ))).
Posted on 7/22/17 11:51 AM in reply to Jason.
gerepad701
Very good blog, we have game recommendations that always get you excited. If you want to find out more, visit our website below.
seo service london
Posted on 8/15/24 12:52 PM.
Audio-Transparency Initiative
Audio-Transparency Initiative