It’s the time of year for saving money!
I had an exchange after a recent Audiophile Review post where a commenter took exception with the idea that to do a “proper” or valid sonic comparison, you must make sure the two sources being compared are at exactly the same output level. And I do mean exactly, measured by some kind of measuring device that gives a quantifiable result. “Close enough” for rock and roll, is not close enough for audiophilia.
Some may consider this an “ugly truth” but it’s actually just the way things are in the real world – often two components that on paper both output 2 volts do not have exactly the same output levels. I’ve measured varying differences between two similarly spec’d components. Sometimes the differences are as little as .1 dB other times I’ve seen .6 dB variations.
FACT: If you do not match the two DACs the one which is slightly louder will be preferred if all other parameters are equal. So, to do a valid and robust test (one that delivers repeatable results) levels must be critically matched. You can, if you wish, object all you like to this fact, but it is still a fact…
In theory, matching levels of two different DACs is simple – measure them and then raise or lower one to match the other one. But the reality can be different. The first issue is that many fine audio components use a stepped rather than continuous volume adjustments. By stepped I mean that each change, up or down, is a specific amount of change. .5 dB is a very common step interval. But what if the difference between components is .2 dB and the steps on the volume control of the louder DAC are in .5 increments. You have the option of having one DAC .2 dB louder or having the other.3 dB softer. Neither will yield a useful result in an A/B test because you simply can’t match the two DACs volume levels. Test fail.
Blind test advocates find “sighted” comparison tests invalid due to the bias of knowing which component is which during testing, but that is a trivial issue compared to doing an A/B test without first matching the output levels of the components under test. This exceedingly low level of rigor flies way past any issues with personal bias and falls right into the bullseye of “invalid test methodologies” that are virtually guaranteed to give a wrong final result.
When I posted on Facebook a very abbreviated version of the previous paragraphs one particular audio fan took offence – he felt I was invalidating all his subjective listening. But as I attempted, unsuccessfully, to explain I was not casting aspersions on the act of subjective audio analysis, but merely trying to make it so when someone does listen subjectively, they aren’t just wasting their time and might, just might, form some valid sonic opinions…
So, if you don’t (or can’t) match your devices’ output levels – don’t bother to do an A/B test, just enjoy the music…
I just bought a new Yamaha universal disc player that has “audiophile” features like high quality DAC, balanced outputs, direct signal path, etc. It incorporates Burr Brown chips. My PS Audio DAC has ESS Sabre chips, as well as multiple digital filters. I have the coax out from the Yamaha connected to the PS Audio unit, and I can easily A/B between the internal DAC vs the PS DAC, but there’s at least a 3dB difference between the levels of the two units (probably because of the balanced connection). Playing about with the volume levels, the two DACs sound shockingly similar! Still trying to decide which sounds best — they both sound pretty darn good! But the initial impression is always that the louder of the two sounds better.
Shockingly similar but perhaps not very shocking. However, I didn’t know that Yamaha makes universal players but I will have to check these out.
Well done, Steve and so true. Many a speaker have been sold simply by being a little louder than the one next to it. When it comes to digital, I’ve even had people tell me they like the mp3 version better. This is including some musicians, simply because its louder on average than the original. Louder will almost always be the choice and in A?B, it needs to be absolutely perfect.
Whenever I hear someone say X “blows away” Y, I ask, “How did you do your level matching?” Then I get the look…