Back, well more than half a century ago, when I was a kid Hi-Fi Crazy of twelve years old, I believed my elders when they told me that the only parts of a "hi-fi set" (which was what we called them then) that made any difference at all were the speaker (back then, all was mono, so there was only one) and the phono cartridge.
Unlike the set's other parts, the speaker and phono cartridge did something that was clearly obvious to anyone at all who would take the trouble to listen: The sonic difference between a radio or TV set speaker and one of the wonders and glories from Bozak, James B. Lansing (now JBL), University, or Electro-Voice (still a huge name in "Pro" audio) was "poke your ears out" obvious, and so was, although perhaps to a lesser degree, the change from one of the ordinary and previously ubiquitous "crystal" or "ceramic" phono cartridge of the day to one of the new and very exotic "magnetic" cartridges (the first of which I knew about being the "variable reluctance" cartridge from GE).
Everything else in the set (or later, when I became a little more sophisticated, the "system") - meaning the turntable, the tonearm, the electronics, and most certainly the cables, was believed to be there just because it had to be; to serve some purpose (the turntable, to turn the record; the tonearm, to hold and position the cartridge over the record; the electronics to make things loud enough to be heard, and the cables to hook it all together), but none of it, we believed, had anything to do with the quality of the sound we heard.
Given the times and the fact that virtually all home audio in those days before 1957 was monophonic (unless you had the ability to play pre-recorded tapes), that was perfectly reasonable: For one thing, it was the official position of the Audio Engineering Society (AES). For another, one couldn't, at the time, simply say to someone "Listen for yourself to decide whether or not this makes a difference" because, even the best equipment of that era was designed and built in such a way as to make most comparisons at least difficult, and sometimes impossible.
For example, the thing that, to my way of thinking, is the greatest achievement of modern High-End audio is its ability to convincingly portray the dimensional and directional characteristics of the performance and the recording venue, but in mono, that simply wasn't possible except in the grossest of approximations. With just a single channel of signal information, the possibility of "triangulating on" the location of any single performer or instrument or on the size or shape of the recording studio or concert hall just didn't exist, so even if an amplifier, for example, (one of the things that the engineers said couldn't sound different unless they measured significantly different) did have the capability of giving a more accurate acoustical "image", there was no signal information available to support it.
Even after the late 1950s, when, with the advent of the stereo LP, stereophonic sound became a national marketing phenomenon, and everybody "had" to own a "stereo", the speakers available to most home buyers were still, for the most part, not capable of producing a believable stereo image. For one thing, the drivers were placed all over the face of the enclosure, wherever it was convenient, instead of in the modern one-above-the-other, or line, or D'Appolito array. For another, it was years before the majority of speaker manufacturers figured out the effects of "edge diffraction" and other enclosure shape factors on a speaker's ability to "image" properly, and time aligning their drivers for that same purpose was still rarely, if ever, done.
In turntables, too, most people still saw only a mechanical necessity, and not a contributor to the quality of the sound. The only things that manufacturers concerned themselves about were "wow"(low frequency speed variation), "flutter" (higher frequency speed variation), and horizontal "rumble" (mostly bearing noise, in the horizontal plane). With stereo records, however, which have a vertical element to their "45/45" cutting technology, and the new "vertical-sensitive" stereo cartridges to play them, vertical rumble became a consideration, too, along with the need for improvement in every other specification, as well.
For one thing, instead of the simple conical stylus shape that had previously been ubiquitous, the styli of the new cartridges were designed to draw more information from LP groove by more closely emulating the shape of the cutting styli used to cut the groove originally and, ranging from elliptical to van den Hul to Shibata and others, they did bring forth more detail, but, in the process, became more sensitive to tracking error and stylus rake angle (also known as "VTA" - vertical tracking angle).
This brought new attention to the tonearm, whether to improve the geometry of the conventional "radial" tonearm (fixed pivot point, cartridge arcing across the record); to attempt some variations on it, including "pantographic " arms (like the B-J, which changed its shape as it traversed the record in order to maintain a "perfect" tracking angle); or to produce actual tangential tonearms like the Rabco, the Eminent Technology and the Clearaudio, which held the playback cartridge precisely tangential to the groove and traversed the record in a straight line, in emulation of the tangential tonearm that held the cutting head that produced it.
The new cartridges were different, too. In addition to being more sensitive to more things, they varied greatly in mass and physical size, and because - all of a sudden - mounting them at the wrong angle or in a tonearm of the wrong mass or with the wrong tracking force could not only result in audibly worse sound, but in low frequency resonances that could, at best, interfere with proper playback and, at worst, actually cause the stylus to be literally thrown out of the groove (!), the importance of matching them to the proper tonearm was finally recognized as being of major importance to the sound of a system. This resulted in more changes in tonearm design, including replaceable headshells of widely differing mass and more available adjustment in height (SRA/VTA), in (for radial arms, only) anti-skating force, and in range of available tracking force.
So; were all those guys right, all those years ago, when they told me that"...the only parts of a "hi-fi set that make any difference are the speaker(s) and the phono cartridge..."? That may have been true at one time, but now that we have better speakers and cartridges, everything else needs to be better, too. Better speakers - ones that have been designed to be capable of stereo imaging and soundstaging - have now allowed us to experience a far better presentation of the spatial characteristics of the original recorded musical event and, if we can hear those on one system, it causes us to wonder why we might not be able to hear it on another and to want to correct that. Same thing with better phono cartridges: Better cartridges allow us to learn the need for better turntables and tonearms; which allows us to learn the need for better phono preamps; which leads us to an understanding of the need for - and ultimately the reality of, better power amplifiers, all of which are displayed and put through their paces by our better speakers.
The fact of it is that "The cleaner the window, the easier to see whatever dirt might still be there" and as Hi-Fi has progressed, that's exactly what has happened: The better our systems get, the more they reveal to us and the more we can be aware of and seek to eliminate the remaining flaws. Our speakers and phono cartridges are still of critical importance to good sound, but now we know that so is everything else.