So between all the classes of amps, all the possible components, solid-state or valve, you're saying they all perform and sound the same...
What I am saying is that amps can be designed well, but some can deliver massive transients better than others, and therefore deliver better sound (both higher detail and more dynamic sound)...
You simply can't say that a Class-D car amp delivers the same quality sound as a Class-A stereo amp (whether integrated or pre/power). Granted the difference is minute compared to source and speaker, but I don't think you can dismiss an entire industry of audio amplification as unnecessary, lol.
I'm not saying they all perform about the same - I'm saying a properly designed amp will sound pretty much identical to any other properly designed amp.

The biggest issue with many amplifiers is that, like you've mentioned, they cannot swing the output voltage/current fast enough to accurately reproduce transient bursts of sound. Assuming they can do that with an appreciably high noise floor and they don't have any weird harmonics because of their design - they should sound pretty much the same as any other good amp.
My real point is that some people swear by tubes, some swear by solid state, but all other things being equal - they'll produce essentially the same sounds within their optimal operating range. Push them too hard and you'll hear the differences pretty quickly.

As for Class D (or T, whatever they call the full-range ones now) versus Class A or A/B - no, I don't think you'll be able to hear the differences as long as there aren't any glaring flaws in either amplifier. Like you said, speaker, room, and source distortion is going to be far higher than any good amplifier.