So this guy comes up to me and says, "Hi Roger, I've just designed this new thing and I'd appreciate it if you'd listen to it and tell me what you think." I'm happy to help him out, of course, but how can I? Can I take his "thing", whatever it may be, and just go listen to it as he asks? Of course not: Whether it's a new speaker or a new source product - one end of the chain or the other - the only way I can actually listen to anything at all is as part of a complete system, played in the acoustical environment of a listening room, or over headphones, and even they have their own acoustical environment issues.
And that's the problem.
Because I can never isolate any one thing that I'm listening to, I can never really tell which part of what I hear or don't hear can be blamed-on or credited-to which part of the chain. I can never even really tell where the chain starts. Is it the original "live" performance? Maybe not: In our modern multi-track world, there may never have been an original "live" performance; instead the music may have been recorded just a couple-out-of-who-knows-how-many-total tracks at a time, with, at first, just a couple of the musicians playing (probably a "rhythm track"),and then other performers may have been added, another couple at a time, with each new batch of performers listening to the performances of all of the previous ones through headphones as they lay down their own couple of tracks, and with the whole thing finally being put together later as a pastiche by the engineer, the producer.
Even if there was a "live" performance, I'm CERTAINLY not listening to it, and I might not even be listening to an "original" or a facsimile of an "original" recording: Most recordings receive some mastering. Depending on what's been done to it, I may or may not like the result, and the odds are that I will never know who did what to it, or why.
What all that means is that even before it gets to my system, the music that I'm going to use to test other things with and judge their ability to convey an impression of reality may not, itself, be "real" at all. That's why I ALWAYS have to listen to multiple recordings and to "piece together" my impressions, instead of relying on any one recording, no matter how good it may seem.
Another reason for doing that is simply that even a perfect recording done in a large hall won't tell me about how well a system will deliver small hall sound and ambience. That was the problem with the old Dahlquist DQ10s. They did big hall BRILLIANTLY, but big hall was all they did: Something about them caused even the most intimate small-club jazz recordings to sound as if they had been taped in Carnegie Hall or on a stage at the Astrodome!
What about female vocals? No matter how good the System is, you're not going to hear what one sounds like on it if there's not one there!
There are other things, too, that, even if they ARE there, can be deceiving: Like the "blattiness" of brass instruments, for example - that hard, nasty, edge that's so much a part of real brass sound, and that's so very difficult for many speakers to reproduce. When I first heard the original Avalon Ascent speakers many years ago, I was absolutely ecstatic! They, for the first time in all my experience, got it exactly right, and brass instruments - whether singly or in chorus - came off absolutely real! The problem, as I found out later, was that the blattiness I was so thrilled with came not from the instruments or from the recordings, but from the speakers, themselves, and EVERYTHING that was played on the Avalons had it, even the woodwinds and that oh-so-elusive female vocal.
So what does all this mean? Well, in the first place, even though everybody loves to play "reviewer" and comment -- hopefully with some degree of knowledge and insight -- on the things that make up or contribute to our hobby, it's not all that easy: For one thing, unless you listen to ALL of what the thing you want to comment on can do, you really (as witness my initial glee with the Avalons) shouldn't comment at all; and for another, because it's hard to tell what part of the System (or even the program material) you're commenting on is the source of whatever it is that you that you find pleasing or otherwise, it's hard to know that what you're saying is true and applicable.
Is there a solution to this problem? I'll have more to say in the second half of this article.