Written by 4:57 am Digital

What Makes a Convincing Audio Review?

Steven Stone wonders why reviewers don’t use more robust A/B tests…


In recent weeks I’ve read several reviews, both pro and amateur, about the newest USB filters from UpTone Audio and AudioQuest. The Regen and Jitterbug are similar in function, but not in internal design. Both offer a way to clean up the signal on the USB path and both seem to work according to all the reviews. So why am I not won over that the devices will make an immediately noticeable positive effect on a system’s sonics? The reviews were not convincing.

AR-convince3.jpgAnd, why did I not find the reviews convincing? After all they were valid A/B comparisons where a reviewer inserted and then removed a component from the signal chain and then listened to both and formed an opinion – how is that not a “real-time” A/B test? Because the time required to do the switchover from A/B was too excessive for their tests to be real-time A/B comparisons. In every case the comparisons relied too on the reviewer’s long-term aural memory.

I have done my share of reviewing computer-based USB DACs and one of the problems with doing A/B comparisons of DACs is that not only must you change inputs on your analog switcher, but you must also change your output device settings, which is an additional step. So you can’t go directly from DAC A to DAC B without a pause of at least five seconds. Inserting and then removing a dongle from a USB chain takes additional time – not only must you attach or detach the cable, but you must then re-specify the DAC as your audio output device since disconnecting the USB breaks the connection which usually results in the computer moving to the next audio device that is still connected. So you must then re-specify your DAC as the audio output device every time you disconnect it’s USB cable. This all takes time. I’m pretty fast when it comes to running such a switch and it still requires at least ten seconds. There’s a lot of time between A and B. And you need to move to make the changeover – you can’t remain in your listening position.

AR-convince4.jpgAnd what would be a more persuasive A/B test? Instead of a component substitution in a single signal chain A/B I would have liked to have seen a dual signal chain A/B. And how and why is this superior to a substitution A/B? It could be set up to deliver a more rapid switch-over from A to B. Sure, a dual signal-chain set-up does require more equipment, but I think it delivers a much higher level of robustness and repeatability to a listening test.

What is needed to do a dual USB DAC signal chain test? You only need one computer, but you do need two identical USB DACs connected with identical cabling. Both DAC’s analog output needs to be switched by the same analog preamplifier. By using two DACs with identical (except for the device in the USB chain under test) signal paths you have a system that lets you switch between them more rapidly with only a momentary lapse in sound. By using two identical DACs you won’t have to adjust the volume between the two becasue they should be the same, and you’ve eliminated one more time-consuming step in going from A to B. Also by using two identical DACs the switchover from one to the doesn not require moving much – The switch, which involves changing the Mac’s MIDI control settings and then changing the preamplifier’s input from DAC A to DAC B can be done without leaving your listening position. I can do this switchover in two seconds.

AR-convince5.jpgDo not misunderstand me, I am not saying that an insertion A/B is invalid or of no value, or that relying on longterm aural memory is wrong, but I do feel that when listening for subtle differences between signal chains a dual-chain A/B is a more robust, reliable, repeatable and convincing testing methodology. And when I read a review of a device that could have been tested that way but was not, I will never be as swayed by the results as much as if they had conducted a dual-chain test.

(Visited 113 times, 1 visits today)
Close