Skip to content

Research Project Update

November 6, 2013

Fuelled by beer and noodles Ash and I met up last night to discuss our proposed research project and start refining our question and approach. Below is a quick summary…

After an extensive literature review we’ve decided to adapt what we are looking at and how. Initial ideas were based around evaluating different surround-sound codecs using either BS1116 (ITU, 1997), or perhaps looking at a range of formats and assessing differences using BS1534 (ITU, 2003). Suffice to say there’s already a lot of research and subjective testing in these areas.

When originally considering formats, we both agreed to include the new Bose sound-bar in any testing. While not strictly a surround-sound system per se, the sound-bar is marketed as an immersive sound experience, and the linear ‘array’ has 5 speakers enclosed and a separate sub-woofer. 5.1 systems have been used in the consumer domain for some time now, and now the sound-bar is attempting to grab a share of this market.

This got us thinking about just how good is it? The logical answer here was to put it up against a 5.1 system in controlled conditions and conduct a comparative subjective evaluation. Bose – for whatever reason they see fit! – appear reluctant to release any technical data regarding either the build of, or algorithms at work within, their product. The apparent lack of subjective research in the field was also a factor in guiding us towards our decision. The research project title has now become… 

“Sound-bar technology versus 5.1: a subjective evaluation of perceived audio differences”.

In terms of methodology we rejected the frameworks of BS1116 & BS1534 as they are too linear, and are concerned mainly with the 1-dimensional concept of quality across different codecs. Their stringent guidelines also focus on expert listeners detecting small differences. We needed a more flexible approach and as result decided that BS1284 (ITU, 2003) was the most suitable framework within which to conduct our subjective test, as “more general assessments usually involve larger differences and therefore do not usually need such close control of test parameters” (ITU, 2003). EBU technical document 3286 (EBU, 1997) was also considered as the data collection was very transparent and subsequent ‘radar graphs’ depicting quality attribute results easy to understand, but the paper stipulates that only classical or acoustic music recorded in a real space can be assessed. As sound-bars and 5.1 systems are used for a variety of audio re-production, this didn’t appear the best fit as it limited our choice of programme material.

In accordance with the recommendation we aim to have either ten expert or twenty non-expert listeners who will be familiarized with the test procedure, environment and material prior to commencing.

As we are currently proposing to use a ‘triangle’ test for comparison, our audio excerpts will employ the 7-grade comparison scale as set out in the recommendation, and our programme material(s) will be selected to “stress the system”, these being a surround-sound recording of classical music, a 5.1 music mix of an appropriate studio recording and audio from a film. We intend to source our programme material from commercially available SACD and either Blu-ray or DVD.

In terms of ‘audio differences’ we plan to finalize our list of attributes at our next meeting, where we will also design our subject input sheet for recording their scores and any comments, which will serve as anecdotal evidence. BS1284 stipulates 7 main attributes to test, each of which has a family of sub-attributes should we wish to drill down. Attribute selection will be confirmed when programme material is decided upon, and subjects will receive an explanation of each attribute and be given an audio example prior to testing. Some examples of main attributes are spatial impression, stereo impression, transparency, sound balance and freedom from noise.

In terms of technical aspects, we propose to conduct the listening tests in Newton, in a room adhering to the standards set out in BS1116 (as defined by BS1284) with the 5.1 speakers configured as per BS775-3 (ITU, 2012). The Bose sound-bar will be set up according to manufacturers instructions and self-calibrated to the size of the room using test-tones.

References:

ITU. 1997. Recommendation ITU-R BS.1116: Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems. Retrieved 6th November 2013, from http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1116-1-199710-I!!PDF-E.pdf

ITU. 2003. Recommendation ITU-R BS.1534: Method for the subjective assessment of intermediate quality level of coding systems.  Retrieved 13th October 2013, from http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-1-200301-I!!PDF-E.pdf

ITU. 2003. Recommendation ITU-R BS.1284: General methods for the subjective assessment of sound quality. Retrieved 6th November 2013, from http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1284-1-200312-I!!PDF-E.pdf

ITU. 2012. Recommendation ITU-R BS.775-3: Multichannel stereophonic sound system with and without accompanying picture. Retrieved 12th October 2013, from http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.775-3-201208-I!!PDF-E.pdf

EBU. 1997. Technical Document 3286: Assessment methods for the subjective evaluation of the quality of sound programme material – Music. Retrieved 6th November 2013, from http://tech.ebu.ch/docs/tech/tech3286.pdf

Advertisements

From → Research methods

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: