Reviews: Who Says It’s Good, And How Do They Know?

In the last installment (Part 3) of this continuing series on audio reviews and reviewers I named three requirements for good reviewing ― the ability to judge products’ performance; the ability to apply some (consistent) standard for rating them; and the ability to communicate one’s judgments and ratings to other people in a way that they will be able to understand and benefit from. I also took a look at the first of those requirements and wrote a little about its own prerequisites: a good enough “reference system” in a good enough room so that all “the good, the bad, and the ugly” of the products under review can be exposed to you; a good enough (but not necessarily “Golden”) set of ears to tell which is what; and enough time spent listening to enough different kinds of program material that you can be sure you’ve heard all about that product that you and your audience will want or need to know.

Once you’ve done the listening, you need to make a judgment of what you’ve heard: Was it, indeed, “good, “bad, or “ugly”? Or was it, more likely, a combination of some or all of those things? More importantly, what’s your standard for judging? What do those things mean to you? You might be surprised to learn just how wildly people’s ideas of what’s good (or what’s anything) can vary.

Once, when I had just started XLO, I got a phone call from a guy who introduced himself as Roy Harris ― at least a namesake of the fairly important 20^th century American classical music composer ― and asked what our cables sounded like. When I told him “Nothing at all”, he asked again, and I responded that “They sound like whatever equipment you have them hooked to, playing whatever music you’re listening to”, and said that, “…to the greatest degree possible, they have no sound of their own.” This still didn’t satisfy him, and we continued a couple of times around and around again with no conclusion until he, exasperated, said “Butterscotch; I like cables that have a warm ‘butterscotch’ sound to them. Is that what your cables sound like?” Finally understanding, I said that no, our cables didn’t sound like that – that their design objective was, in fact, to have no characteristic sound of their own at all – and suggested that he try some other brand.

Now, consider this: If he had been reviewing cables, and had been asked to review some from XLO, would he have liked them? No!  And if I had been reviewing the “butterscotch-sounding” cables that he was seeking, would I have thought they were good? Of course not! Different people have different ideas of what’s good, and, like politics or religion, everyone thinks that his own is the only one that’s correct!

So is there some kind of an “absolute” standard that everybody can agree on? Well, yes and no; everybody agrees with the late Harry Pearson that the ideal hi-fi product would be one that, when played,  would sound exactly like the music as it was originally performed, “live”, but just exactly what does that mean? Do you have to be a musician or a recording engineer to know what music really sounds like?

Lewis Lipnick was (and may still be) the first-chair contra bassoonist for the National Symphony orchestra (of Washington, D.C.) and was also a reviewer for Stereophile magazine. Does being a concert musician make him a better reviewer?

Frankly, I have no clue, but I do know that his idea of what classical music sounds like is bound to be different from mine: As a bassoonist, he always sits on the stage, surrounded by all of the other players, to the right (the audience’s right; his left) of the center of the orchestra, and toward the back of the stage, just in front of the trombones and other brass instruments. I, on the other hand, when I’m at a concert, sit, as I must, in the audience, in front of the orchestra, with all of the instruments facing me, from whatever distance and to whatever degree right or left of center that the price and availability of my ticket has allowed.  What that means is that not only the soundstage and orchestral “image” that Lewis Lipnick and I hear will be different, but that the tonal balance and even the hall reverberation effects that we hear ― EVEN AT THE SAME CONCERT ― will be completely different, too.

So, is his perspective on what the music is supposed to sound like better than mine? What if it’s music of some kind other than classical?  If neither of us performs music of a particular genre, is the fact that he’s a musician and I’m not of any advantage at all to him or disadvantage to me in reviewing hi-fi equipment? 

And what if there was no actual performance at all? What if, instead of a concert recording, the music being played for reviewing the equipment is a “studio” recording, done in multiple takes, on multiple tracks (with, perhaps, not even all of the performers ever in the studio at the same time), mixed together and “mastered” later, possibly even by some other engineer than did the original recording, in some other studio, different from that where the original tracks were recorded? What, in short, if there was never any actual “performance” at all? How’s anybody supposed to compare the sound of music played on a system including equipment under review to the sound of “live” music, when there was no live music to begin with? 

Would a recording engineer know better? With no original live session, how could he?

Back in the days when Tom Miiller and I were both reviewing for Sounds Like… Magazine, we came up with an answer that, while perhaps not perfect in every application, was at least good enough for us at the time: What we listened for and judged equipment by was “believability” – the degree to which the recording, the system, or the particular piece of equipment under review could cause us to believe, correctly or not, that we were listening to live music played for us by live musicians.

Obviously, that’s not always going to be the same for all people: Different people will give their belief to different things, depending on their culture, the kind and range of their experience, and on their pre-conceptions of what they think a thing “ought to” sound like. (Consider the old RCA recordings or the Memorex and other ads where a dog [Nipper] is claimed to be fooled by the sound of “his master’s voice” or people, as early as the beginnings of the twentieth century, claimed, on hearing recordings that we might find unlistenable today, to be unable to tell the difference between live and recorded music) Even so, the “belief” standard seems to be about as good as we’ve got, and – if we and a reviewer happen to share the same belief, we’re going to think he’s right every time.

I’ll go into this and other things more in the next installment.

See you then!