Search Search

Testing methodology

Actually sound quality evaluation test through the internet is a very complicated task. Many recommendations (by ITU-R/AES/EBU) for listening tests could not be fully implemented in the internet testing. Listening sessions have to be easy enough for non-prepared listeners and results must be firm against various listening environments and skills of those ordinary listeners.

Taking all this into account only the core of SoundExpert testing engine was based on traditional testing methodology - double blind, A/B scale with hidden reference or in ITU-R terms - double stimulus / hidden reference / double blind methodology. For tested devices with clearly audible artifacts the method is used as is. The only difference is absence of the third stimuli - non-hidden reference as it could be picked out from two samples with ease. Participants download a sound file with two samples of the same sound excerpt. One of them is a reference sample and the other is processed by device under test. Testers have no information either about order of samples or about device used. Listeners are asked to point to processed sample and to grade its Basic Audio Quality (BAQ) according to standard five-grade impairment scale.

For tested devices with hard to distinguish artifacts this methodology is used in combination with sound artifacts amplification (SARTAMP) – special technology aimed to gain artifacts to some controlled level. In this case the whole testing procedure is completely the same except computing of final ratings. They are calculated considering the gain applied to artifacts.

Detailed explanation of SE testing engine and method can be found in the papers below (in the order of increasing complexity).

1. SoundExpert. How it works. (PDF: 240k)
Easy to understand overview of SE functionality.

2. Infinite Grade Impairment Scale, Smirnoff S (PDF: 175k)
The paper describes briefly new method of judging audio quality - the one SoundExpert is based on.

3. DIFFERENCE LEVEL. An objective audio parameter. Smirnoff S (PDF:694k)
The paper describes objective parameter, called “Difference Level” that could be considered either as an extension of THD for non-periodic signals, or as one of the estimations of widely used difference signal. It could be used for instrumental measurements of signal degradation in various audio circuits and for psycho-acoustic research. Infinite grade impairment scale and corresponding method for measurements of perceived audio quality, based on this parameter is also proposed.