Skip to Content

What is SoundExpert

Audio metric that makes a differenceSoundExpert (SE) is an independent non-commercial audio research project, which provides audio quality ratings of sound equipment and technologies such as mp3, aac, wma ... encoders and portable players (to be extended). It is the home for innovative music-based audio metric which works both for objective audio measurements and for listening tests. SE is in beta state because the new audio metric and corresponding listening test design are not widely adopted by audio engineers. So, please, use our ratings with caution.

The new listener-centric audio metric allows to perform objective audio measurements using real-life music material and various technical signals, not only sine waves. Such measurements:
• correlate well to human perception of sound quality
• are valid for digital audio
• do not contradict traditional audio metric but extend it
• are understandable to people without technical background
• can be performed at home using any recording device

The first practical application of the new measurement method can be found on the Portable players page.

The new listening test design helps to hear sound artifacts which are normally beyond threshold of human perception. The core of the new design is sound artifacts amplification technology (SARTAMP), which is backed by the new audio metric. Such amplification is particularly important for testing high-bitrate encoders and high quality audio equipment. This technology is used in SE listening tests since 2005. The tests are blind - participants don't know the particular device they assess. Resulting quality scores are completely unbiased and free from marketing and advertising noise.

Live Listening Tests @SoundExpertThe ratings of encoders are computed in real time while participants of the listening tests send their grades to the rating system. The more participants - the more devices and technologies can be tested. The testing procedure is simple and short enough that anyone, including you, can participate. Just download a test file (<3Mb), listen it (<15s) and send back your grade. Details are in the test file. The more accurate grades - the more reliable ratings.

Detailed but still easy to understand overview of SE distributed listening tests is in the paper Live Listening Tests @SoundExpert.

* * *

The main idea behind SoundExpert project is consumer control over sound quality of various audio equipment and technologies on the market. Production side deserves clearly defined feedback from consumers.

 

 

Articles

« Back

Audio Quality of High-End Portable Players

Audio Quality of High-End Portable Players


This article is a result of collaboration with csglinux - Head-Fi forum member. The cooperation was so successful that we decided to extend it - to perform df-measurements of portable players with much more accurate measurement instrument. For the purpose we assembled DIY recorder based on TI PCM4222EVM. It is capable of recording music signals with ~-80dB precision, which is sufficient for most portable audio devices. The new df-measurements can be seen either on HypetheSonics web site [https://www.hypethesonics.com/dapti-database/] or in the SE Vault [http://soundexpert.org/vault/dpa.html].

This page is preserved for archival purposes.
 

 

csglinux testing setup

Testing setup of headfi member csglinux.

 

Thanks to csglinux we have df-measurements of these high quality portable players:

The list is expanding, it's limited only by the good will of the contributor and reliability of his RME Babyface Pro. The latter has better specs than SE recorder, so these df-measurements can't be just added to the portable players page [the research needed]. But we can compare them separately below. The one-page manual "How to read df-slides" is here → http://soundexpert.org/portable-players-beta2#howtoreaddfslides

Df-slides below are sorted by histogram median. Top df-slides represent portable players with higher playback fidelity.

Audio measurements of Chord Hugo 2
          Chord Hugo 2: This is how excellent audio performance looks like - three groups of test signals and 1-bit signal alone.

Audio measurements of Shanling M0
          Shanling M0: Df-measurements of this player were checked several times. No mistake.

Results of LG V30 audio measurements

Results of LG V40 audio measurements

Audio measurements of Questyle QP1R

Audio measurements of iBasso DX220

Audio measurements of Astell&Kern SP2000

Audio measurements of Apple Lightning Audio Adapter
          Apple Lightning to 3.5mm Headphone Jack Adapter: The only small problem is the jitter visible on Sine 12.5kHz diffrogram (vertical greener lines).

Audio measurements of Apple iPhone SE

Audio measurements of Apple iPhone 5

Audio measurements of FiiO M11

 

 

Artifact signatures of the tested devices

In df-metric artifact signatures of devices can be computed and compared with each other. So, audio devices can be grouped according to similarity of their artifact signatures. Below is 3D visualization of such groups for the tested players. You can zoom and rotate the image.

 

The portable players in similarity space. The closer devices to each other, the more similar their artifact signatures. Colors of devices correspond to Df values of their histogram medians. The distance (mean-corrected MAE) of 2 dB is critical for relation of Df measurements to subjective scores. For players that have close artifact signatures (distance < 2 dB) their perceived sound quality correlates well to df-measurements performed with substantial amount of real-life music material (histogram medians).

 

Comments
Adrian
I can't believe this never occurred to me before: does the Df-metric capture stereo crosstalk as another possible defect of reproduction? I notice it says "L+R" on the diffograms next to anything involving complex signals, but is that a summation performed _before_ applying the Df-metric or is it symbolic of some logarithmic mean between two Df-metrics computed separately for the L and R signals? I suspect the former option would mask rather than reveal crosstalk, and make fidelity seem higher than it is for devices with bad crosstalk.
Posted on 3/29/23 12:07 AM.
Audio-Transparency Initiative
Audio-Transparency Initiative