Published Date 7/7/15 11:27 AM
Four years ago SE started sound quality evaluation of portable media players. Results collected for the first few contenders showed that testing procedure has some serious drawbacks and needs to be corrected. For example, using real headphones as a load (and especially Koss Porta Pro) leads to specific coloring of sound for players with higher output impedance. Also, recent research related to diffrograms revealed that test files for analog audio devices can be and must be processed with higher accuracy. And finally, objective measurements of analog circuits have sense. New method of objective testing, based on Difference level parameter, helps to discover problem signals and to evaluate waveform degradation of real-life sound material for any device under test. So, results of such testing in addition to usual SE listening tests would be helpful.
Taking all this into account it was decided to start testing of portable media players afresh. Previous results will be discarded, sorry.
The first contender for this enhanced testing is iPhone 6. Results of its objective measurements are summarized in compact form - df-slide, shown in Fig.1. All df-slides for portable players which were tested at SoundExpert are collected on this page - http://soundexpert.org/portable-players-beta
Figure 1. Results of testing media player of iPhone 6 with various audio signals and single measurement approach based on Difference level parameter.
The diagram shows how various signals (including real-life ones) are distorted when they are played back by iPhone 6. We can safely conclude that:
- pure sine waves both 1kHz and 12.5kHz have low levels of distortion (-79.0dB and -78.9dB)
- inter-modulation distortion is pretty high (the sum of sine waves 12460Hz and 12540Hz at the output of iPhone 6 has substantially higher level of difference with initial waveform: -45.9dB)
- serious gap between accuracy of reproduction of triangle and square waves (-38.7dB and -16.3dB) means that iPhone 6 has phase distortion; signals with multiple frequency components are affected the most (white noise -4.6dB)
- median of 6430 Df values (the whole Pink Floyd album divided into 400ms pieces) sums up all possible types of distortion of the real-life audio signal; the value (-25.7dB) is pretty close to degradation of Program Simulation Noise (-24.9dB); this is very promising and even a bit surprising; level of degradation of this signal could be a single integral and informative objective indicator which says a lot about performance of a device under test; more audio devices should be tested in order to make more confident conclusions about usefulness of such indicator.
Other test signals could be synthesized and used for discovering various aspects of an audio device performance. As the measurement procedure is the same for all signals, their values of degradation can be compared with each other directly; the whole audio metric becomes connected.
Feel free to share and discuss the above diagram.
SoundExpert thanks Rasim for allowing to test his iPhone 6.
SoundExpert reminds you that final conclusion about audio quality of any device added can be made only after finishing corresponding listening test. So we encourage you to take part in testing. SE ratings exist only thanks to visitors like you. Testing is short and easy, visit our Testing Room for simple 1-2-3 instruction.
Please, remember, downloading test files from SoundExpert you will not necessarily get the ones of the newly added devices or codecs. SoundExpert testing is blind testing – you don't know the device you test. This is the only reliable way of getting true audio quality ratings, free from any cheating and various human senses other than perception of sound quality. After you have sent your grade you will see the device you tested.
Update 2015 Aug 26
After adoption of final set of test signals (http://soundexpert.org/test-signals) it was decided to test iPhone 6 once again. Results are below.
Figure 2. Results of testing media player of iPhone 6 with final set of test signals and single measurement approach based on Difference level parameter.
- triangular and square waves are band-limited now, so they are reproduced with higher accuracy (lower Df values) by device under test
- triangular (even) signal is added (ass train); comparing its degradation with the standard triangular (odd) wave we can conclude that even harmonics slightly prevail over odd ones at the output of iPhone 6.
SE thanks Olesya for allowing to test her iPhone 6.