SoundExpert

Rating bar graphs improved

2010-07-25T07:34:57Z

Some time ago a visitor of SE site by the name of Uhl, who accidentally turned out a human factor engineer, pointed out some shortcomings of our rating bar graphs – extending left beyond the first score and disproportional lengths of bars relative to their values. He suggested a few variants which eliminate those drawbacks. We found the arguments are reasonable and corrected our bar graphs according to the recommendations.

Old rating bar design

New rating bar design

Thank you, Uhl, for your efforts; new bars are visually more correct and understandable indeed.

Micropayments by Flattr

2010-07-22T19:48:50Z

Another micropayment system - Flattr appeared on February 2010. It was founded by ex-PirateBay Peter Sunde and entrepreneur Linus Olson. The main idea is to give people the possibility to say “Thank You!” for any thing they like on the internet with 1-2 or even a fraction of a cent by simply pressing a button. The “thing” can be a single blog post, a song, a photo, a whole web site – any piece of creative work. Much like the “Like” button on Facebook but supported with money.

In order to thank people this way you have to register with Flattr and upload minimum 2EUR to your account. Each time you press the Flattr button on a web page a fraction of this amount will be transferred to the author of the thing you like. In order to get your own Flattr button you have to register with flattr and upload … - exactly the same way because Flattr doesn't discriminate between creators and consumers of content. In modern life we are both the same time. Then you register your content in your account and get the button code. Details are explained on Flattr web site.

All things already registered on Flattr are available for discovering in the catalog. On the 15-th of July it contains 17907 things. They are divided into 6 sections:

Written text 81.3% (14469)
Images 5.8% (1031)
Audio 4.9% (866)
Video 2.7% (487)
Software 2.1% (370)
The rest 3.3% (584)

Let's have a closer look at Audio section. It numbers 866 things. Each day 10 more audio things are registered on the average. As told before, a thing can be either a single creative work or a bunch of them. Both tactics are used now by Flattr participants (we at SE decided to register the whole site). Please, bear this in mind looking at how things fall apart in Audio section:

Podcasts 65.1% (564)
Indie/Recognized musicians 21.8/0.5% (189/4)
Web radio 5.3% (46)
Music blogs 2.1% (18)
Net labels 1.6% (14)
Videocasts <1% (9)
Non-audio <1% (7)
Music pieces with unclear authoring <1% (6)
Artist aggregators <1% (5)
About Audio <1% (3)
Audio books <1% (1)

Predominance of podcasts, covering various topics from love stories to software engineering, seems a bit surprising. Top 10 most flattered things are also podcasts. But if we think of podcasting as a next step in personal blogging and a very convenient form of “broadcasting” both for program authors and listeners, those figures become more understandable. Conventional radio is trying to be more convenient for listeners and listeners convert their blog posts into podcasts.

Indie musicians, Net labels and Aggregators (offering web space and services to artists) is another big group which benefit from Flattr micropayments. Almost all musicians published their works under Creative Commons licences. Among recognized musicians are:

Carbon Based Lifeforms
Sian Chandler and Ray Hughes
Pandemonium
Ian Simmonds

Only first two of them have Flattr buttons on their official web sites. Carbon Based Lifeforms accompanied donation buttons with the following: “If you downloaded our music on p2p or otherwise and still want to pay us, feel free to donate”. Now with Flattr button it is extremely easy to do this. Ian Simmonds and Pandemonium don't mention Flattr on their official web sites, so the question - who will receive microdonations for them – is open. Seems like the only solution for such uncertain situations is using Flattr buttons on official web sites or other places reliably controlled by authors.

Web radios presented in Flattr catalog mostly rotate music under CC licenses by indie-artists. One discovery is worth to mention – SceneSat Radio playing “the best music from every corner of the scene and related stuff” as they say about themselves. We could add that streaming content has surprisingly good artistic and technical quality (professional pre-processing, four streams: aac+ vbr@48, mp3 cbr@128, aac+ vbr@128, mp3 cbr@320), not to say that the station has very convenient way to say “Thank You!” with Flattr button.

Six things point to music pieces which are difficult to understand whom they belong to and who will receive the revenue. Eighteen things are from music blogs. They raise the question - is it ethical to ask for donations for discovering new music and promoting artists? Well … why not?! Presence of Videocasts and ordinary blog posts (16 in total) in Audio section looks like mistake. Three things belong to web resources devoted to Audio itself as a technology (SE – one of them). While numbers of things submitted to Audio section changes rapidly and when you read this article actual figures will differ from those above, the structure of submissions will stay actual for longer time as it changes more slowly if at all.

At the moment Flattr is still closed beta – you need invitation to join. Hope when it goes public there will be substantial increase in content submissions which will help Flattr to get critical mass – a key factor of any micropayment service success. By the way, we have three more invitations; drop us a line if you need one.

ATRAC SP (Type-R) in 320 kbit/s group

2010-07-17T13:02:49Z

First hardware audio codec - ATRAC1 SP (Type-R) has been just added to SE. Some of the reasons why:

For relatively long period of time MiniDisc and ATRAC SP were a standard of hi-fi digital recordings at home and in semi-professional area.
The format still has substantial user base consisting mostly of demanding listeners.
Its perceived audio quality is well known, discussed and compared a lot with other formats.
Development of ATRAC stopped and now it could be considered as Latin among lossy audio compression schemes.

Taking all these into account ATRAC SP will be a perfect anchor point on SE quality scale, helping other ratings of upper end to be more meaningful.

ATRAC SP is also a landmark in the history of SoundExpert. For the first time output signal for a codec was obtained during real-time recording-playback on hardware device - MD recorder. Usually real-time recordings have small time inconsistency, which is insignificant for most tasks of processing/listening but easily destroying difference signal - the core of SE artifacts amplification technology. Thanks to precise time warping procedure developed inside SE those time inconsistencies can be effectively corrected now with predefined accuracy. This special time-treatment opens SE for wider class of devices/technologies which now can be rated. Also for the first time test recordings for a codec were made outside SE by volunteers. The whole story is in two news articles:

As usually SE thanks several people who contributed their time and efforts to make all this possible. Lee Keywood for this codec proposal and invaluable help during development of time warping procedure. Numinos and Mike Rofone for making test recordings on their MD equipment.

Now ATRAC SP is in the system along with a hundred other codecs. First reliable results are in two-three weeks depending on testers' activity. Participation is simple and easy as usually.

Cheers,
SE Team.

Analysis of submitted ATRAC SP (Type-R) recordings

2010-06-23T22:19:14Z

1. Collection of submitted recordings

All submitted recordings were processed with time warping algorithm and listed in the table below.

Author of submitted recording	Difference levels (dB) of time warped SE samples (Left and Right channels)																		Download
	BAH		BAS		CST		FMS		GLK		HRP		LOB		MOF		QRT
	L	R	L	R	L	R	L	R	L	R	L	R	L	R	L	R	L	R
Lee1	-38.1	-38.7	-41.1	-40.0	-19.4	-19.7	-35.0	-34.9	-30.3	-30.7	-19.9	-19.9	-26.2	-22.6	-27.3	-27.0	-37.1	-36.2
Lee2	-38.5	-39.2	-40.9	-39.4	-19.3	-19.8	-34.7	-34.9	-31.6	-31.2	-19.8	-20.1	-25.7	-22.0	-26.6	-26.7	-36.8	-36.3	RAR 65.5Mb
Lee3	-38.5	-39.0	-41.3	-40.5	-19.5	-19.5	-34.8	-35.2	-31.1	-31.2	-19.7	-19.8	-26.0	-22.3	-26.7	-26.6	-36.6	-36.2

Numinos1	-37.3	-37.7	-39.2	-38.8	-18.9	-19.2	-34.2	-34.3	-29.5	-29.3	-19.3	-19.5	-25.1	-21.7	-26.0	-26.0	-36.3	-35.5
Numinos2	-37.8	-38.0	-39.8	-38.9	-19.2	-19.1	-34.6	-34.4	-30.1	-29.9	-18.6	-19.7	-25.0	-21.8	-27.0	-26.6	-36.3	-35.5
Numinos3	-37.6	-38.3	-39.4	-38.7	-18.6	-19.3	-34.5	-34.6	-29.2	-28.8	-19.7	-19.3	-25.9	-22.5	-27.0	-26.7	-36.4	-35.6	RAR 65.6Mb

MikeRofone1	-31.6	-31.7	-32.8	-32.7	-18.1	-18.4	-30.9	-30.9	-30.0	-28.7	-19.3	-19.4	-24.1	-21.2	-25.1	-24.9	-31.7	-31.5	RAR 65.0Mb
MikeRofone2	-31.6	-31.7	-32.7	-32.6	-18.3	-18.6	-30.9	-30.8	-26.5	-28.1	-19.1	-19.5	-24.5	-21.3	-25.1	-24.9	-31.6	-31.5
MikeRofone3	-31.6	-31.7	-32.9	-32.7	-18.2	-18.5	-30.9	-30.7	-25.8	-24.9	-19.2	-19.3	-23.8	-20.5	-25.1	-24.4	-31.7	-31.4

MikeRofone21	-38.6	-38.9	-41.1	-39.9	-19.2	-19.4	-34.6	-34.8	-33.7	-31.9	-20.1	-20.5	-25.5	-22.2	-27.2	-26.7	-36.9	-36.4
MikeRofone22	-38.5	-38.8	-40.6	-39.4	-19.4	-19.6	-34.5	-34.7	-30.0	-30.4	-19.8	-19.9	-25.8	-22.2	-26.7	-26.5	-36.9	-36.3	RAR 65.3Mb
MikeRofone23	-38.2	-39.0	-41.3	-39.5	-19.3	-19.4	-34.7	-34.5	-27.1	-25.9	-19.8	-20.0	-24.8	-21.5	-27.0	-26.8	-36.8	-36.2

Notes:
1. Difference level shows how much does the warped signal differ from reference one.
-Infinity(dB) means identical signal shapes,
0(dB) – completely different shapes.
2. As time warping procedure is computationally extensive fast version of the algorithm was used for processing of all submitted recordings. The candidate chosen for inclusion to SE rating system will be processed with the highest possible accuracy. Accurate time warping gives slightly lower values of Difference levels (around 1dB lower) and lower contamination of processed signal with time warping artifacts (noticeable in the above difference signals as amplitude modulation).
3. Test file used for recording consists of three identical groups of SE test samples. Difference levels are indicated for all of them. Time warped versions are uploaded for one of them only as they are almost similar.

2. Cluster analysis

Here is graphical representation of Diff. levels from the table above:

All recordings except MikeRofone1-3 look consistent with each other. However the graph doesn't show how similar they are. Cluster analysis is more informative in this sense.

This dendrogramm shows how the recordings break up into groups according to similarity to each other. The higher the |¯|-shaped lines the less similar connected recordings. In our analysis all recordings could be divided into three groups:

Lee1-3 and MikeRofone22 are most similar. This is not surprising because Lee did some research to find the setup which provides the lowest number of errors while recording-playback-recording on his equipment. MikeRofone22 also hit the group and MikeRofone21 is the second closest. However MikeRofone21-23 recordings are less similar to each other.
Numinos1-3 recordings are also less similar to each other but pretty close to the first group.
MikeRofone1-3 recordings differ greatly from all others. They have to be excluded from further analysis.

3. Selection of best samples

Recordings from group#1 - Lee1-3 and MikeRofone22 plus MikeRofone21,23 - were time warped with higher accuracy. Results are below:

Author of submitted recording	Difference levels (dB) of time warped SE samples (Left and Right channels mixed)									Download DIFF signals
	BAH	BAS	CST	FMS	GLK	HRP	LOB	MOF	QRT

Lee1	-39.42	-43.25	-22.11	-37.44	-33.60	-21.10	-27.12	-30.81	-37.44	RAR 9.7Mb
Lee2	-39.88	-42.84	-22.06	-37.65	-33.45	-21.01	-26.85	-31.33	-37.40	RAR 9.7Mb
Lee3	-39.80	-43.69	-22.05	-37.57	-34.13	-20.83	-26.58	-31.06	-37.21	RAR 9.7Mb

MikeRofone21	-39.76	-43.22	-22.01	-37.52	-34.01	-21.56	-26.40	-31.19	-37.36	RAR 9.7Mb
MikeRofone22	-39.65	-42.73	-21.97	-37.34	-33.39	-21.48	-26.95	-30.98	-37.36	RAR 9.6Mb
MikeRofone23	-39.64	-43.09	-21.82	-37.40	-32.46	-21.54	-26.50	-31.16	-37.27	RAR 9.7Mb

Note: It turned out that Diff. level of mixed Left and Right channels is more sensitive parameter because it also accounts phase shifts between channels. From now on it will be used instead of separate Diff. levels.

Visual representation of the values:

Audibly similar type of artifacts in diff signals and close values of Diff. levels clearly show that all recordings belong to Type-R compression. The only difference between them is number of recording errors. In other words degradation of the signals caused by two factors - ATRAC compression and recording errors.

Unfortunately each author's recording has only three observations for each sound sample. This doesn't allow us to perform analysis of variance which would reveal distribution type of those observations. So we assume that it is one-sided normal distribution because it is highly unlikely that recording errors could improve Diff. levels over those conditioned by ATRAC. If so we could chose for each sound sample the recording that has the lowest Diff. level of warped signal, which means that the recording is less affected by recording errors. These lowest values are marked with green in the table.

The selected recordings were chosen as the best representatives of ATRAC SP (Type-R) encodings of nine SoundExpert samples. They were then time-warped with accuracy below -100dB - thus hiding artifacts of time-warping beyond resolution of 16 bit audio. Resulting Diff. levels, diff. signals and time-warped samples are in the table:

Author of
submitted
recording

Difference levels (dB) of time warped SE samples
(Left and Right channels mixed)

Download

BAH

BAS

CST

FMS

GLK

HRP

LOB

MOF

QRT

Lee1

-22.28

-27.10

-37.46

diff
(RAR:9.7Mb)
warp
(RAR:25.5Mb)

Lee2

-39.90

-37.69

-31.38

Lee3

-43.82

-34.16

MikeRofone21

-21.62

4. Contributors and equipment

SoundExpert would like to thank people who recorded SE test file on their equipment:

Lee Keywood (www.leekeywood.co.uk)

Recording chain: SE test file(.wav) ==> PC(Linux) ==S/PDIF==> Sony MZ-R410(SP,Type-R) ==MD-disc==> Technics SJ-MD100 ==S/PDIF==> PC(Linux) ==> Lee1-3(.wav)

Numinos from Minidiscforum.de (forum post)

Recording chain: SE test file(.wav) ==> PC ==USB-S/PDIF==> Sony MZ-NH600(SP,Type-R) ==MD-disc==> Kenwood DM-3090 ==S/PDIF==> Sony MZ-NH600(PCM) ==upload as PCM through SonicStage==> PC ==> Numinos1-3(.wav)

Mike Rofone from Audio T-board (forum post)

Recording chain: SE test file(.wav) ==> Audio CD ==> Pioneer CD player ==Toslink==> Sony MZ-RH1(SP,Type-R) ==upload as Hi-SP through SonicStage==> PC ==> MikeRofone1-3(.wav)
Recording chain: SE test file(.wav) ==> Audio CD ==> Pioneer CD player ==Toslink==> Sony MZ-RH1(SP,Type-R) ==upload as PCM through SonicStage==> PC ==> MikeRofone21-23(.wav)

Call for ATRAC SP (Type-R) test recordings

2010-05-09T21:34:35Z

From now on SoundExpert is ready to accept wider class of devices/technologies for testing and rating. Hardware audio equipment for example. This became possible owing to development of precise time warping procedure which eliminates various time distortions of audio signals usually caused by AD/DA conversion, filtering, inaccuracy of clock rate, etc. Being inaudible and practically immeasurable such time/phase inconsistency affects greatly the difference signal used for artifacts amplification in SE testing scheme. Fortunately all software codecs produce time accurate output while most of other devices/processings usually have constant or variable (wow-and-flutter) time shifts. Now we can precisely adjust time grid of an audio signal to time grid of reference signal. Thus difference signal contains only shape differences regardless of time squeezing/shrinking.

The first device that will be added to SE rating system using this kind of “time treatment” is hardware codec ATRAC SP (Type-R). It has no software implementations and used in MD players/recorders/decks. Recording/playback of ATRAC SP streams as many other real-time processes is affected by inaccuracy of clock rates. Resulting digitally recorded streams usually have small variations of their time scales, not audible but sufficient to spoil difference signal completely. So ATRAC SP is a good candidate for testing our new time warping technology.

We thank Lee Keywood who not only suggested inclusion of this codec for benchmarking purposes but also encouraged us to develop the time warping procedure. He carefully made on his MD equipment tens of recordings that we used for testing and tuning of our code.

This successful collaboration convinced us in possibility of preparing testing material outside SoundExpert by volunteers. Corresponding methodology of selection and verification of submitted output samples was also developed and now needs to be tested in practice.

For the purpose we ask volunteers who have MD equipment with ATRAC SP (Type-R) capability to record digitally SE test file and then record it back to PCM format digitally as well. This is in short. In detail:

Download a single test file - http://rapidshare.com/files/381711351/secd01.rar (WAV, 44100, 16, stereo, duration 5:15, size 29.7 MB). It consists of three identical groups of nine SE test samples (with trailing samples). Three groups are necessary to verify the consistency of recoding. The test file can be played back on computer directly or by mounting virtual Audio CD. Also it can be burnt as real Audio CD and played back on CD/DVD player. CUE-sheet for burning/mounting applications is included.
Using S/PDIF, TosLink, AES, USB or other digital interface record the test file to MD (Type-R) as a single track.
Playback the recorded MD track through digital interface and record it back to computer or other recording equipment capable of recording PCM 44100. If you record in 32bit format, leave it as is to avoid additional processing.
Archive or losslessly compress resulting WAV/AIFF 44100 PCM file and upload it to some file hosting service.

As we never did this before we don't know how many such recordings will be sufficient (this is another goal of the experiment). We'll process all received files with our time warping software to eliminate wow-and-flutter and other time distortions. For every file submitted we'll upload corrected (warped) versions along with difference signals. So the whole experiment will be openly covered on SE site and could be discussed on Sony Insider forum.

The most consistent and representative recording will then be added to SE rating system as ATRAC SP (Type-R) device under test. Although this format is not developed anymore it is still used for home and semi-professional recording. It will serve as a benchmark for SE rating scale, let alone the possibility to compare it to other audio compression algorithms at various bit rates. Also if this experiment is successful the above scheme could be used for adding other audio equipment to SE rating system.

Thank you in advance,
SE Team.

New Nero AAC Codec 1.5.3.0 in 192 kbit/s group

2010-03-09T11:24:18Z

The latest version of free Nero encoder has been just added. According to SE site statistics 192 kbit/s group is the most visited now. So it will be populated with contenders first of all. Thank you, Taras, for the proposal.

Cultural evolution @home

2010-01-03T11:52:40Z

One more Distributed Human Project appeared on the internet. DarwinTunes models cultural evolution using music as an example. The project assumes that there is an analogy between darwinian evolution of species and cultural evolution inside human society, that all cultural diversity exists thanks to continuous copying and modifying of previous cultural artifacts and subsequent selection and preserving of good modifications. For the purpose DarwinTunes continuously produces new generations of music excerpts – loops by modifying the best ones from previous generations. Selection of good ones is entrusted to visitors of the site. Everyone, including you, can participate. You will be asked to express your attitude towards generated music excerpts choosing from “I love it!”, “I like it”, “It's OK”, “I don't like it”, “I can't stand it!”. At the moment more than 60 000 visitors took part in the experiment and we can see\hear music loops after 300 generations of evolution.

If you think the goal of the experiment is creation of some musical masterpiece, you are mistaken. DarwinTunes aimed to research the process of evolution itself, how it develops in time, what factors affect it. The music evolving inside the project is pretty interesting, though.

Besides the main goal of the project there is another one, may be even more important – to show that cultural evolution progresses by means of copying and modifying. So in order to support progress and creativeness we must make available as many objects of culture in public domain as possible. It wouldn't be a big exaggeration to say that promoting such seditious ideas DarwinTunes moves cultural evolution further, not just researches it.

Open DarwinTunes in new window >>>

Vorbis Xiph(1.2.3) vs. aoTuVb5.7 @80kbit/s

2009-12-26T20:49:14Z

It was correctly mentioned by visitors of the site that Ogg Vorbis encoders at SE are a bit outdated now. The more so as there were some developments in this field during 2008-2009. Vorbis encoder became an audio part of Theora – free and open format for multimedia exchange on the internet. Xiph.org foundation which passionately favors the spreading of this open source format, released their version of the encoder – libvorbis 1.2.3. It is used now for encoding in VLC player, RealProducer, multiple command-line utilities and applications for all operating systems. It is a part of Theora package. The same time thanks to Aoyumi who contributed a lot to make this codec sounding better, we have the latest and the most tuned version of the encoder – aoTuVb5.7. Probably all the enhancements made by Aoyumi will be integrated into Xiph's “official” version in some future, but for now the two major versions coexist peacefully along with four or five less widely used ones.

In order to find out how the “official” Xiph version compares to other non-Vorbis encoders and to see how far the latest development by Aoyumi goes, we decided to include them both:

libVorbis 1.2.3 and
aoTuVb5.7

For making this comparison fair a small research was conducted. Below are bit rates of the above encoders depending on -q parameter which sets quality level of encoder output. Nine SE samples are used.

High resolution copy of the chart available in PDF(306kB).

It's clearly seen that the curve consists of several separate monotonous segments indicating stepwise changes in psy-models of the encoders. And their comparison would be more informative for both developers and general public if it is within the bounds of corresponding psy-segments. So the first contenders for SE testing were chosen:

aoTuVb5.7 -q1
libVorbis 1.2.3 -q1.75

They use the same psy-model segment and result in the same target bit rate – 80.4 kbit/s. This quality setting (-q1) could be recommended for streaming purposes. And it perfectly fits SE 80 kbit/s group .

Also all Vorbis entries in SE ratings were renamed according to Xiph example – OGG used for media container and VORBIS – for audio codec.

As usually we thank people who suggested the codec for inclusion - Norz, Morten Wendelboe, Francis Chabot.

Don't forget to take part in testing!
SE Team.

SoundExpert 3.0

2009-12-14T01:00:05Z

The third version of SE testing system is here. What's new:

Open source portal engine Liferay is used as a basis for SE project now. It means that such collaboration tools as forums, wiki, polls, blogs and others could be easily integrated into SoundExpert.
Thanks to Konstantin Pupkov code of testing engine was revised and rearranged according to portlet architecture of Liferay. The logic of SE testing remained untouched. Some parts of testing engine were optimized for performance and two bugs were fixed:
- if there were three or more devices on the page with the same rating, only two of them were displayed.
- because of mistakes in MySQL records a few test files have never been downloaded and graded. But because of existence of two versions of each test file (with different position of original) the deviation of affected ratings is inessential and will be corrected naturally with the lapse of time.
After sending back of a grade participants will see results of their listening test including device they tested.
IP addresses of all submitted grades are recorded now
Distribution of test files was switched from FTP to HTTP. For the sake of simplicity and robustness we dropped resume option while downloading. The feature is completely senseless in this particular case as all test files are small and faceless. Just download another one, it doesn't matter.
We are .ORG now. This domain better reflects the nature of SE project – ratings by people and for people. So the canonical name is http://soundexpert.org/ The old domain - .INFO will be alive as well.
You'll be surprised but we became beta. While SoundExpert testing methodology is still not confirmed by independent parties (all interested in are welcome) the beta state of SE research project will indicate that you can use the ratings at your own risk. We do believe the method will be recognized and approved by audio engineers but until then SE will be marked as beta.
And finally, we moved to new server. Usual story, you know …

While working on the new version we didn't add new codecs to the system. All suggestions received during that period will be considered once again and some poll might be organized on the topic. The very next contender for testing is Ogg Vorbis. As usually fill free to contact us with your suggestions, bug reports, critics, ideas, etc.

By the way, testing engine is fully operational already, you can try it right now in Testing Room.

SoundExpert is moving to a new location

2009-12-09T19:22:53Z

New version of SoundExpert testing system is ready and will be launched in a few days. In order to preserve integrity of all collected data (your grades) SE testing engine was stopped. It will be resumed after upgrade of the system. Ratings will be displayed as usually, not updated though. We'll do our best to make this transition as seamless as possible.

Nero AAC Encoder 1.3.3.0

2009-12-11T11:45:39Z

A few days ago the new version of free Nero AAC Encoder (1.3.3.0) became available. According to its change log there are no significant improvements affecting quality of stereo encoding though. But taking into account that previous version (1.1.34.2) didn’t participate in SE testing at all and any version of this encoder never tested at SE at low bit rates where it usually shows the best efficiency we decided to perform extensive testing of the new encoder starting from low bit rates (thank you, Lance Penguin for this proposal). First four settings added are VBR: -q 0.15, -q 0.20, -q 0.25, -q 0.35. They dropped into the sections respectively: 32 kbit/s, 48 kbit/s, 64 kbit/s, 96 kbit/s. We intentionally did not tune resulting bitrates with q – parameter but exercised commonly used anchor values instead. Fine-tuned contenders can be added later if needed. As usually first reliable results will appear in a week or two. Thank you all for participation in listening tests and Vito Marolda personally for donation.

Comment: antk

The previous version of Nero AAC encoder was not tested in the 192kbit/s setting. How about adding it there (as well as in the 224kbit/s section) ?

Lame 3.98 (-V 0) added

2009-12-05T10:20:43Z

Next setting of new Lame mp3 encoder (3.98) to be tested at SE is “–V 0” – the highest quality VBR mode of the encoder. Resulting in 231.9 kbit/s on SE test files it fell into 224 kbit/s group. As usually first reliable results are expected in a week or two. But you can help SE to improve some of its green (grey) ratings faster - just take part in testing … if you want.

Lame 3.98 in 128 kbit/s group

2009-12-05T10:22:36Z

We must thank developers of Lame mp3 encoder for their time and efforts - Lame 3.98 released to public. In order to compare the new encoder not only with other encoders but also with previous Lame versions SE will add new Lame with the settings which are the same as or similar to the ones already used in Lame SE testing. It will depend on resulting bit rates produced by new Lame on SE test samples, though.

First setting is "-V5" - joint stereo variable bit rate mode resulting in 131.3 kbit/s on nine SE test samples. So it perfectly fits 128 kbit/s group at SE. Previous version (Lame 3.97b2) when using this setting (-V5) showed lower bit rate - 112.7 kbit/s and its comparison with other contenders producing around 128 kbit/s output was not very honest to be true. Now with 131.3 kbit/s the competition will be more fair.

Next setting planned for testing is "-V0" - highest quality VBR mode. It was also used for testing of Lame 3.97b1 at SE. If you have some other suggestions, please, let us know.

Thanks for participation in testing.
The People from SE

Bluetooth audio quality (A2DP)

2009-12-05T11:24:17Z

Bluetooth audio devices become more and more popular today – stereo headsets and headphones, wireless audio transmitters/receivers, cell phones and mp3 players with Bluetooth Advanced Audio Distribution Profile (A2DP). Headphones and headsets seems benefit the most from Bluetooth (BT) audio. To be honest first BT stereo headphones were of poor sound quality – dirty sound with annoying “hiss” on background. Definitely that was not because of BT technology limitations but mainly because of poor implementation of the latter and slipshod design of analog audio circuits. The situation changes slowly but steadily. For example modern BT stereo headphones DR-BT50 from Sony can successfully compete with ordinary wired ones already. Their sound quality will be sufficient for vast majority of portable player owners for sure. In no doubt if manufacturers decide they could produce perfectly sounding BT headphones for demanding listeners as well. In that case the way digital audio transmits over BT protocol is a bottleneck in the whole audio transmission chain.

As the data channel used by A2DP is only 721 kbps wide some data reduction scheme is required. While BT specifications allow using of different audio codecs (mp2, mp3, wma, aac and even atrac) the only mandatory codec for all BT audio devices is subband codec (SBC). In fact many BT headphones (including above mentioned DR-BT50) already support mp3 codec but in real life it remains unused. Obviously it was made for the sake of compatibility but the same time portable devices could benefit from direct streaming of mp3 files through BT interface saving both audio quality and battery life. However in today's practice all high quality audio still transfers over A2DP by means of SBC codec.

SBC codec appeared in SoundExpert ratings more than two years ago (see 320+ kbit/s section). The bitrate used for testing (372 kbps) showed good potential of this compression scheme but it’s clear now that above setting doesn’t reflect real-life scenario of the codec use. Let’s look for example at these two major BT applications:

Toshiba Bluetooth stack for Windows (6.10) has three quality modes for SBC codec

High Quality – 328 kbps
Middle Quality – 229kbps
Low Quality – 201 kbps

IVT Corp. Bluesoleil (5.0.5) has two quality modes: High and Middle. It’s not mentioned what bitrates are used but the application has special “Status Window” available for any BT connection. Among other information it shows amount of bytes sent to and received from connected device. Simple calculations reveal the same bit rates for High and Middle quality modes – 328 kbps and 229 kbps. As Bluesoleil is capable of both sending and receiving A2DP audio data its status window helps to discover actual bitrates used by any connected BT device. For example cell phone Nokia 6500 classic also sends music to BT interface at 229 kbps and the bitrate can’t be altered in phone’s settings (may be some advanced music phones and players have such possibility, though).

Such unanimity in choosing SBC codec settings is not surprising. Bluetooth Special Interest Group (SIG) recommends using of those settings in A2DP specifications (A2DP_SPEC, Revision V12):

Table 4.7: Recommended sets of SBC parameters in the SRC device
SBC encoder settings*	Middle Quality				High Quality
SBC encoder settings*	Mono		Joint Stereo		Mono		Joint Stereo
Sampling frequency (kHz)	44.1	48	44.1	48	44.1	48	44.1	48
Bitpool value	19	18	35	33	31	29	53	51
Resulting frame length (bytes)	46	44	83	79	70	66	119	115
Resulting bit rate (kb/s)	127	132	229	237	193	198	328	345
*Other settings: Block length = 16, Allocation method = Loudness, Subbands = 8

Taking all this into account it was decided to add SBC codec to SE testing engine with these new settings:

SBC 229kbps (Middle Quality) – in 224 kbit/s section
SBC 328kbps (High Quality) – in 320+ kbit/s section

It’s worth to add that A2DP limits the available maximum bit rate to 320 kbps for mono and to 512 kbps for stereo modes allowing the use of many other codecs besides SBC. So audio manufacturers (of headphones especially) have wide choice of compression technologies capable of delivering high definition sound through the wireless digital channel at reasonable cost.

AAC VBR@256 from iTunes

2009-12-09T13:02:01Z

New codec in 256kbps section – AAC VBR from iTunes. Now both CBR and VBR modes of iTunes AAC codec @256kbps are under test. Definitely VBR mode consumes a bit more bits for encoding SE test files. Does this lead to any quality improvements? We’ll see. SE thanks Bill Justin for this codec suggestion.

eAAC+ @48 kbps from Winamp 5.53

2009-12-09T13:14:21Z

Low bitrate codec eAAC+ @48 kbps widely used for uploading music to mobile phones is added. Thank you Gert (aka Snoyt) for the proposal.

HE AAC 64kbit/s codec from Winamp added

2009-12-09T13:51:46Z

HE AAC codec from Winamp (aac he, cbr, 64 kbit/s) was added to SE testing system. Low bitrate sections of SE are a bit outdated at the moment, so starting with this codec we plan to fix the situation soon. SE thanks Giulio Franco for this codec proposal.

New iTunes AAC added

2009-12-09T23:35:07Z

Finally AAC encoder in iTunes was updated. The new one is included in iTunes 7.4 shipped with new Mac OS Leopard. Starting from version 7.5 iTunes for Windows also has this AAC coder. The bug of previous release is completely gone and resulting bit rates of encoded files are a little bit lower. That’s all to be said for sure in the beginning of testing it at SoundExpert.

The testing is started with CBR@256kbit/s setting which is used in iTunes music store. It looks like this bit rate will be the most popular soon. In a week or two first reliable results will show new encoder potential. If you want to get them faster, download and grade a test file from SoundExpert. Please, remember, the test file that will fall to your share will not necessarily belong to the newly added codec, it can be any of 95 codecs rotated in SE at the moment. Blind testing is blind testing. Thank you in advance for participation.

WMA 9.2 VBR(Q90) added

2009-12-13T21:06:28Z

Wide support of WMA format along with MP3 by manufacturers of portable players and powerful marketing strategy of MS did the job – WMA is very popular.

Q90 setting was added to 192 kbit/s group although it renders SE test items at average 160 kbit/s bitrate. If there are some other contenders in this bitrate sub-group, new page "Coders 160 kbit/s" will be inserted.

In case of current activity of volunteer testers first reliable values for the rating will appear in a fortnight. SE thanks Stig Lykke Iversen for this codec proposal.

Try to grade one of the 94 codecs rotated in SE system at the moment (ZIP: ~3Mb)

SoundExpert system in simple words

2009-12-09T23:26:46Z

New article explaining SoundExpert functionality in simple words appeared on Testing methodology page. Authors would be thankful for your feedback here.

SoundExpert. How it works (PDF: 236k)