It’s the time of year for saving money!
There are certain subjects in high performance audio that are certain to get lots of people excited. Few get audiophiles more riled than the efficacy of Double Blind Tests (DBT). Publish an article proposing the notion that DBT aren’t necessary and you’ll be subjected to a litany of criticism and usually accompanied profane insults.
The idea of a DBT seems reasonable and logical. Participants don’t know the product or products being tested and neither does the person conducting the test. In this way, the proponents claim, an accurate appraisal may be enlisted across a wide cross section of people about whether or not the product being tested lives up to the claims made by the manufacturer. On the whole, I’d have to say it all sounds very reasonable. On paper.
I’ve talked with a number manufacturers and have, at times, asked them about DBT. Almost universally, they are of the opinion that the main reason they don’t conduct them publically is once the product has been released for production, the product has met the design goals. At that point, a DBT isn’t necessary and consequently is up to the consumer to decide if the product is worthwhile. They also told me, again almost universally, that if a separate entity wanted to conduct a DBT using their product, they certainly had no objections. Again, that sounds very reasonable.
I beleive there would be little or no disagreement with the idea that the best way to determine if you like an audio product is to test it in your system. I can think of no better method to evaluate a component or cable’s performance than doing so with equipment you already own and are intimately familiar, with music you’ve heard countless times, and in a room whose sonics are completely understood – and that would be your own personal audio system.
Yet in a recent article I wrote about a comparison between High Rez and Standard Definition digital files there were quite a number of comments in full support of DBT. I was accused of being too old to accurately hear the music, of having cognitive bias, and falling victim to the placebo effect – and all written by people who have never met me or heard my system. That sounds like a fancy way of telling me I’m a liar. And for the record, I emphatically maintain I heard a difference on MY system. But that’s getting off track.
Many of the comments to that article supported DBT and claimed that if one was not used, then no accurate results, including mine, could be reasonably confirmed or justified. I also found it interesting that of the several respondents who actually make recordings not only agreed there was a difference, they basically dismissed DB testing in favor of listening as the best and final determination of any differences between standard rez and hi rez files. Who would you rather trust to repair your car – an auto mechanic or someone not trained to work on an internal combustion engine? It seems therefore reasonable to place higher value on a professional recording engineer’s opinions when considering resolution questions.
I once was watching a series of comments on an article about a particular speaker cable take place. There were a handful of people that essentially praised them – most of the respondents did not. The author of the article had gloriously reviewed the cables. The respondents gloriously trashed them. Soon enough, the issue of DBT arose.
I watched all these comments go back and forth and finally, I wrote to ask the person with the most objections if he had ever heard the cables in question in his system. He replied that he hadn’t, but he was confident that because of the cost, and that a DBT had not been conducted, they couldn’t possibly work. That is like saying you hate cottage cheese despite having never tasted cottage cheese. At that point, I lost interest.
If we can agree there is no better method for an individual to decide on how a particular component works in their system than testing it in their system, then does a DBT really matter? If I am to judge something I may want to purchase for use in my system and find out if, (a) the manufacturer’s claims are supported, and (b) if I like the component or not, if I am to be my own final word on what I like best, then a test performed by strangers seems counterintuitive. At best, a DBT will be performed by people with a wide range of listening skills, with a system of unknown quality and performance compared to mine, in a venue with different acoustics and with unfamiliar music. This is supposed to be a preferable method to an in home test with known variables? Frankly, I just don’t see how that is possible. For someone to tell me I’ll like the sonic performance of a product in which there are so many unknowns, in my view, is completely unsupportable.
One possible exception to this would be if someone was actually a participant in a DBT and then tested that same component in their system. I’d say that scenario would be reasonable. On paper. However, what happens if the component performed better or worse in the home system than the DBT? If that happened, the the DBT test is basically pointless.
The issue of DBT will continue to be divisive, I realize that. Personally, I’ll continue to do my own evaluations in my system as the last and final barometer of a particular component’s worth. I’ll use my own listening skills, flawed or not, as my judgment. For those who disagree and support the whole idea of DBT and are completely devoted to such testing, that is also perfectly fine. Discord makes debate possible. Differences of opinion and preferences in various things makes the whole hobby a tad more interesting and that makes things more fun.
In the end, I don’t have the right to tell someone who supports DBT they are wrong for doing so. But that position also works both ways.
