Monday 2 April 2012

Are blind tests bogus? Examples of blind tests with positive results.

Some claim that blind tests are bogus because they are designed to create a fail or cannot be passed and they involve trickery or deception.

I do not think that is true because

- they can be passed, such as blind tests of speakers and bit rates. Use the same test with cables and the result is a fail. That is because there is no difference between cables, but there is one between speakers and bit rates. If the test was itself designed to fail, then why not fail with speakers?

- I do not like the blind tests that have been done where people are told they are listening to different cables, but in fact there has been no change. It is interesting, but again a bit dubious when something like a wire coat hanger is slipped into a cable test without anyone's knowledge. My preference is the simple two cables, let the subject see and hear both in action. Then once blinded use one and then say, 'I am may or may not change to the other cables now, please say if you can hear a difference or not?' Then repeat that for about 20 times. Where is the deception in that?

You can also decide if you do have 'golden ears' here

http://www.audiocheck.net/blindtests_index.php

So here are blind tests that have positive results where people could hear a difference.

1 - A blind test of speakers, passed by the subject. Interestingly, the subject failed to identify different crossovers, one more expensive than the other.

http://www.audioholics.com/news/editorials/axiom-blind-listening-test

2 - One of amps with 500 participants EDIT - it is debatable whether this is actually a pass or not.

http://www.stereophile.com/features/113/index.html

3 - Power amp blind tests, two of which are positive.

http://home.provide.net/~djcarlst/abx_data.htm

4 - Head-Fi, by member Pio2001 between a Marantz integrated amp and Pro-ject Headphone amp

http://www.head-fi.org/forum/thread/429619/headphone-outputs-lots-of-measurments-and-one-abx

5 - An interesting Boston Audio Society article about two tests. The tweaked CD test is a fail, but read on and an amplifier blind test is a pass.

http://www.bostonaudiosociety.org/bas_speaker/wishful_thinking.htm

6 - A Hydrogen Audio test of different gauge speaker cables.

http://www.hydrogenaudio.org/forums/index.php?showtopic=14082&st=25

7 - PSB speaker blind test, the top of the range speaker won

screenOvenNRC.jpg

http://www.psbspeakers.com/audio-topics/Birthplace-of-Good-Sound

8 - ABX Comparator. A series of blind tests of different kit and cables.

Starting with the cables, differences were found with video cables over very long runs of 100 feet in comparison to a 6 foot one.

http://home.provide.net/~djcarlst/abx_vid.htm

Then interconnect and speaker cables, five tests and no differences found.

http://home.provide.net/~djcarlst/abx_wire.htm

A speaker test with a very large sample found 97% could tell the difference

http://home.provide.net/~djcarlst/abx_spk.htm

CDPs and a DAC did less well

http://home.provide.net/~djcarlst/abx_cd.htm

Power amps did a bit better

http://home.provide.net/~djcarlst/abx_pwr.htm

But what was very noticeable was the likes of distortion, filters and a small change in volume

http://home.provide.net/~djcarlst/abx_data.htm

9 - Matrix Hifi between two amps where two testers got all 30 tests correct, 60 attempts in all (in Spanish, Google translator used)

http://translate.google.com/translate?js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&sl=es&tl=en&u=http%3A%2F%2Fwww.matrixhifi.com%2Fcontenedor_trivsclat.htm&act=url

10 - 16bit vs 24bit, Gearslutz.com forum, 9 out 10 correctly identified

http://www.gearslutz.com/board/so-much-gear-so-little-time/358301-24-vs-16-bit-not-audible.html

11. Sampling rate, 44.2kHz vs 88.2 kHz, AES May 2010

http://www.aes.org/e-lib/browse.cfm?elib=15398

No comments:

Post a Comment