When we try to ascertain sonic quality of something we have three basic options - yes, that sounds better, no, that sound worse, or it sounds the same. We can add layers to those evaluations, such as "this one has better bass response but the other one was better in the midrange." All in all, making decisions on what's better, or less so, is one of the most difficult decisions in audio.
It is universally accepted that in home demos are perhaps the best way to critically evaluate something. Any room where an audio system is housed is a significant composite of the sonics the system will produce. So much so, one system may sound markedly different in one location than another. How many times has a component sounded great at a dealer and at home, just didn't light things up in the same fashion? Probably lots of times. How does an audiophile then reconcile sonic value when two vastly different presentations exist?
What about time? How much time should be spent to make an honest, informed opinion? It is not at all uncommon for consumers to be expected to make a decision on something at the dealership. And if the whatever may be borrowed, it should be returned in a couple, maybe three days - seldom beyond a week. This speaks nothing of buying over the Internet and the sonic uncertainty of doing so.
When making a decision, listening is supremely important. Yet, how one product sounds will quite commonly sound different than another. We all have our own likes and dislikes. I spent at least five years chasing a "sound" I wanted out of my system. It took quite a few different components, speakers, cables and even a different home for me to finally find satisfaction.
Most puzzling is that if during those five years you asked me exactly what I was looking for I wouldn't have been able to tell you. I would have said "I don't know, but I'll know it when I hear it." It could also be realistically argued I am still chasing that sound. No two people are necessarily enthused by the same thing. Likes and dislikes may be quite varied. I was once at an audio show demo where different tonearms with different wires were being shown. The point was, the tonearm with the better wires inside would sound better. I was sitting beside a reviewer for a popular magazine and after the "better" was played, I asked what he thought.
He didn't hear any difference between the two. I thought the version with higher quality wires actually did sound better. Therein lies the rub, as they say. I liked one option more than the other and he didn't. Does that especially mean one of us is right and the other wrong?
Does his opinion mean more because of a pedigree? Or are there no real absolutes, only differences of opinion?
Many of us turn to evaluations based in some part on measurements. Well, why not confuse things even more, right? It's difficult enough making a decision on something based only how how it sounds, now I'm supposed to include scientific measurements in my evaluations?
I am in no way negating the importance of measurements. They will always be important. I have an app on my iPad with a spectrum analyzer, takes FFT readings and shows dB levels and I use it quite often.
However, I don't use any of that to help me decide on something, I use it mostly in support of a decision I've already reached. Now, of course, my iPad app is not on the same level as calibrated, laboratory grade equipment will be, but my needs are not the same.
It does, however, raise the question of what worth are measurements given. I'm sure most audiophiles have heard some version of a popular saying - "if it measures bad but sounds good then you've measured the wrong thing." Yeah, there are any number of variations of that oft used dictum, but the idea is the same. How do we really decide on which has the most importance - sonics or measurements? If both tell the same thing, then fine. What happens when they are polar opposites?
As years go by, perspectives change. How many remember the incomparable importance in the 1970s given to "total harmonic distortion" or THD for short. How many amplifiers didn't sell very well because their THD was 0.01 as opposed to a competitor being 0.0001? THD was so supremely important back then it was almost an exclusive feature. I remember completely ruling out any number of products, in those days mostly amps, only because I didn't like their THD. Today, we have better learned how to manage distortion. For example, how many owners of low wattage SET tube amps could care absolutely less about potential high levels of distortion? They love the sound. Distortion doesn't matter.
Some of us, inexplicably, turn to forums. Choose any of the popular audio forums and ask this question: "I'm looking for a really great amp with excellent bass response that costs no more than $2000.00. Any suggestions?"
Do that and watch the recommendations pour in like the sea. Really, can any measure of an informed decision be made that way? Is it wise to realistically trust someone who is completely unknown, not to mention a completely unknown audio competence? Yet it happens every day.
How do we then answer this question? What, exactly, is the best way to decide sonic worth? What is the best methodology to use when so many discrepancies exist? At the end of it all, it is a very personal matter, one we must make for ourselves through research, testing and critical listening. We all have that "little voice" inside our head telling us we like this or don't like that.
And yes, that's essentially a polite way of saying it's just a crap shoot. Much like vegetables, some will like it one way, others will like something completely different. It's really just that simple. Or complicated.