Sonic Comparisons - If the playing field isn't level the conclusions will be lopsided

AR-sonicDBS.jpgI had an exchange after a recent Audiophile Review post where a commenter took exception with the idea that to do a "proper" or valid sonic comparison, you must make sure the two sources being compared are at exactly the same output level. And I do mean exactly, measured by some kind of measuring device that gives a quantifiable result. "Close enough" for rock and roll, is not close enough for audiophilia.

Some may consider this an "ugly truth" but it's actually just the way things are in the real world - often two components that on paper both output 2 volts do not have exactly the same output levels. I've measured varying differences between two similarly spec'd components. Sometimes the differences are as little as .1 dB other times I've seen .6 dB variations. 

AR-sonicC5a225.jpg

FACT: If you do not match the two DACs the one which is slightly louder will be preferred if all other parameters are equal. So, to do a valid and robust test (one that delivers repeatable results) levels must be critically matched. You can, if you wish, object all you like to this fact, but it is still a fact...

In theory, matching levels of two different DACs is simple - measure them and then raise or lower one to match the other one. But the reality can be different. The first issue is that many fine audio components use a stepped rather than continuous volume adjustments. By stepped I mean that each change, up or down, is a specific amount of change. .5 dB is a very common step interval. But what if the difference between components is .2 dB and the steps on the volume control of the louder DAC are in .5 increments. You have the option of having one DAC .2 dB louder or having the other.3 dB softer. Neither will yield a useful result in an A/B test because you simply can't match the two DACs volume levels. Test fail.

AR-sonicC3a225.jpg

Blind test advocates find "sighted" comparison tests invalid due to the bias of knowing which component is which during testing, but that is a trivial issue compared to doing an A/B test without first matching the output levels of the components under test. This exceedingly low level of rigor flies way past any issues with personal bias and falls right into the bullseye of "invalid test methodologies" that are virtually guaranteed to give a wrong final result.

When I posted on Facebook a very abbreviated version of the previous paragraphs one particular audio fan took offence - he felt I was invalidating all his subjective listening. But as I attempted, unsuccessfully, to explain I was not casting aspersions on the act of subjective audio analysis, but merely trying to make it so when someone does listen subjectively, they aren't just wasting their time and might, just might, form some valid sonic opinions...

So, if you don't (or can't) match your devices' output levels - don't bother to do an A/B test, just enjoy the music...

comments powered by Disqus

Audiophile Review Sponsors