A couple of years ago when the first sound pressure meter apps became available for the iPhone and iPod Touch I downloaded one and started using it to measure the volume levels during demos at shows. I also used it to see what the volume levels were during my listening sessions at home.
At the first Hi-Fi show where I used the SPL meter app I immediately noticed that, on average, the playback levels I was experiencing were at least 5 dB and often 10 dB louder than my listening levels at home. The reason that the SPLs were higher at shows was pretty obvious - the background noise levels were much higher, usually measuring between 50 dB and 70 dB as a base level. At home my listening rooms measure between 35 dB to 40 dB base background level. That's at least a 15 and possibly as much as a 30 dB difference.
The maximum peak SPLs I've measured in my home system were approximately 100 dB. My loudest average listening level at home is usually somewhere between 90 and 95 dB. At hifi shows I've measured average levels above 100 dB and peaks over 110 dB. That is loud - so loud that I pull out my earplugs from my pocket a stick 'em in my ears.
Obviously at home, because of the far quieter background noise levels, I can listen at a lower level than at shows. And while some music, such as a Mahler symphony, demand a higher volume level than a recording of a solo mandolin, even "big" pieces of music rarely get played where the maximum level exceeds even 95 dB. Louder doesn't add, but instead detracts, from my enjoyment.
These high ambient noise background levels at shows are one of the reasons that many systems don't sound as good at a hifi show as they do in a dealer's showroom or at home. At a show a system is often forced to work well outside its comfort zone and the resulting sonics indicate that. The extra loudness comes at the cost of added distortion and non-linearity.
Playing at a high volume level stresses an audio system and can drive its components past their optimum operating range, but high volumes also stress a human's internal sound reproduction system. Just as excessive input levels can drive microphone preamps into distortion, your ears can be over-driven by external sounds. And the more frequently your ears are abused, the more sensitive they become. Just like an electrostatic panel that has been arced once by too much power, your ears will ring more and the ringing will last longer after each dose of high level sound till someday the ringing never stops completely. Hello, Tinnitus...
And what is the takeaway from this? You can, if you aren't careful, easily damage your hearing by consistently listening music at too high a volume level. You can also insure that your sound reproduction system will perform sub-optimally by playing music too loudly.
And while I would never try to tell anyone what level they SHOULD listen to music at, I do encourage you to find the volume level that delivers music without stressing your system or your ears.
Ideally, loud shouldn't be painful to you or your stereo system...