I tend to agree. I'll also point out that it's partly an issue of language. I feel like (at least in the case of English) the words for describing taste are less developed than, say, sights.
So while I can describe a car finish as "matte black", or "iridescent blue-green" without resorting to simile, I can't describe flavors without saying something like, "it tastes like {another thing you may have also tasted}". So wine tasters end up leaning on seemingly wacky comparisons to communicate the subtleties they perceive (real or imagined).
I always thought that "notes of blank" was kind of silly until I saw this flavor wheel for coffee [0], and it clicked that tasters simply had a vocabulary that they tended to draw from because flavors are hard to describe.
I can totally relate. For me, it "clicked" when I saw a TV show where one of the moderators went to a tasting session and tried to learn how to taste wine. They showed an aroma wheel similar to this one: http://www.regalwineco.com/sites/all/themes/newregalwineco/e... The aroma wheel helps to categorize the different tastes, and to give them names.
It's really hard to describe what you're tasting if you're not experienced. I'm not experienced at all, I just know whether I like a wine or not, and I can describe it as "heavy", "fruity", "dry" or "light". But the aroma wheel and the vocabulary helps when trying to taste lighter nuances. And sometimes you take a sip, and are instantly reminded of berries, or of oak wood :)
Does it matter? I don't know. In the end, I either like a wine or not. But if you're interested in a topic it helps to share a vocabulary to describe your experiences.
Some color names like "orange" are derived from object names. For tastes can we say that bitter, sour, salty, sweet are the primary colors? Not sure about scents. For audio we named the notes. For textures under touch we seem to have independant names.
So while I can describe a car finish as "matte black", or "iridescent blue-green" without resorting to simile, I can't describe flavors without saying something like, "it tastes like {another thing you may have also tasted}". So wine tasters end up leaning on seemingly wacky comparisons to communicate the subtleties they perceive (real or imagined).