Standardized CT Rating system?

CellarTracker Main Site

Standardized CT Rating system?

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [Cellar Talk] >> CellarTracker Support >> Standardized CT Rating system?

Page: [1] 2 next > >>

Message

<< Older Topic Newer Topic >>

Standardized CT Rating system? - 1/28/2007 3:46:04 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

I've spent some time browsing around the forums and FAQ here looking for some guidance on Ratings and I haven't turned up much. Google has been surprising unhelpful as well. I found the obvious links here to Parker's 50-100 rating system and a vague description of how he says he rates wine, but it still seems overall that there isn't an awful lot of consistency in the community ratings (which is to be expected when a diverse group of palates and experience come together.

I'm somewhat new to trying to be serious about wine, but I've learned a lot and am trying to learn more. The whole system of ratings really seems to be more of an inexact science than it should be; different experts have their own methods of rating and I'm SURE folks here have different ideas about what makes up a rating. Half of the professional reviews I read make me think they write commentary like Simon Cowell reviews American Idol contestants -- expereinces the product and whatever mental picture is formed becomes the review regardless of whether or not it makes sense.

My biggest shortcoming are being able to describe what aromas and flavors I'm experiencing and not having sufficient breadth (e.g. what is a Lugana SUPPOSED to taste like?) and depth (e.g. this is the BEST Pinot Noir I've had, but how would it compare to what the 'perfect' 100pt Pinot Noir would be?) to properly assess a GOOD wine.

There's also a concern about QPR (Quality:Price Ratio) coming into play in a rating score combined with a compression of rating scores in the 80s. IMHO, a rating should be independant of cost, but it seems that some CT reviewers bump up the score because a wine is "90 -- an awesome $8 everyday wine!!!" while others rate it poorly because the wine is "70 -- rank, undrinkable; what you'd expect from a $8 wine." (an exaggeration of this, a wine I'd rate in the mid/high 80s: http://www.cellartracker.com/wine.asp?iWine=48353).

I'll stop rambling now and get to my point. If CT would like users to emulate a Parker-type scoring system (his devotion of 10 points to 'aging potential' makes scoring, um, interesting), I would suggest that CT revise its scoring section on the Drinking Notes to require the level of detail Parker says he uses. I put together an Excel doc that does help me more objectively score a wine. CT could calculate the final score based on the components and present the summary score and summary notes in the "Tasting Notes" section with a link to the detail breakdown (both in summary and review-by-review detail) should users want to see where the summary scores came from.

An example of my Excel form looks like this (for a Barefoot Cab). In my version, I 'hide' the summary score so I can't see the final until I highlight the field.

I would also suggest giving rating tips to improve the consistency of each category, such as:

Attribute
Suggested Score_____Characteristics

Color / Appearance
05_____Perfection. There is nothing about the visual experience of this wine that could possibly be improved.
04_____
03_____
02_____
01_____
00_____Major appearance defects (milky, completely off color)

Nose / Bouquet
15_____Perfection. The aroma from the glass is full and complex. The nose is distinct to the varietal and sets the standard against which all other wines of the variety are judged.
12_____
09_____
06_____
03_____
00_____

Flavor (10) & Finish (10)
20_____Perfection. The flavor and finish of the wine is representative of what all wines in the category should be judged against.
16_____
12_____
08_____
04_____
00_____

Aging Potential
10_____A once-in-a-lifetime wine that is expected to continue to improve and develop for 20+ years.
08_____Shows characteristics that indicate this wine will continue to improve through the the next 5-10 years.
06_____Shows characteristics that indicate this wine will continue to improve through the the next 2-5 years.
04_____Wine is near it's peak potential. Expect only modest improvement in the next 1-2 years.
02_____No cellaring potential. Drink now, this wine will not improve.
00_____This wine has long since peaked. Characteristics have become muted and will only decline.

< Message edited by Corrado -- 2/2/2007 8:20:16 PM >

Post #: 1

RE: Standardized CT Rating system? - 1/28/2007 5:41:58 PM

Serge Birbrair

Posts: 3344
Joined: 4/23/2006
From: Boca Raton, Florida
Status: offline

I refuse to rate wines like a professional. It's their job. I don't see anybody here trying to do what dentists or surgeons do, professional wine critisism is also a job requiring certain knowledge and experience most of us don't have.

When I throw the number, it represents overall impression, something I can do,
but more and more I use binary scoring system:
1- I'd buy it again
0 - I can live without.

Works for me.

_____________________________

La Tache is for amateurs.

(in reply to Corrado)

Post #: 2

RE: Standardized CT Rating system? - 1/28/2007 6:25:47 PM

kilsen

Posts: 31
Joined: 1/6/2007
From: Philadelphia, PA (USA)
Status: offline

Corrado,

You raise some excellent points, regarding the quality and consistency of wine ratings that come from such a diverse community. I think that your suggestions for standardization may help; the question is, how will they be received by people who are already accustomed to "scoring" wines?

Personally, when tasting wines, although I've taken notes regarding color, bouquet, and taste, and I've reached conclusions as to whether I would want to drink (or purchase) more of the wine, I have not yet assigned a numerical score to any wine I've tasted. I simply don't consider myself "qualified" to do this, and I don't consider my palate sophisticated enough for my scores to be trusted -- even by myself!

Nevertheless, I feel that I want to develop the ability to rate wines consistently and reliably -- and this includes being able to provide full verbal descriptions of the wine, as well as a numerical score. There are a few reasons for this desire. For one thing, I'd like to be able to trust my own personal ratings and descriptions -- and so I'd like to be able to do more than give a wine a thumbs-up or a thumbs-down, as it will help me to better understand the KIND of wine I like, and why I like it (so that I can more successfully identify similar wines that I am more likely to enjoy once I've tasted them). And I'd like to be able to look back at a wine rating that I wrote months or years previously, and still be able to understand why I wrote it, and why I scored it the way that I did (and ideally, the description should evoke my sensory memory of the actually experience of tasting the wine). In other words, well-written wine reviews should be a "conversation with myself" across time.

But I want to do more than keep my tasting notes private, because participation in the CellarTracker community needs to be RECIPROCAL. Obviously, I "trust" a Parker or Tanzer review much more than the review of any random CT user, and I "trust" professional scores more than the aggregate scores of the CT community. Still, I am reading tasting notes written by other CT members; I am looking at individual scores assigned by other CT members; and I am (in part, at least) judging wines that I may or may not taste/purchase based on the aggregate scores from the CT database. Thus, I honestly believe that I have an OBLIGATION as a member of this community to contribute to this knowledge base.

As an aside, it must be said that even professionals' ratings and scores can vary significantly. Earlier in my wine-drinking experience, I was using Wine Enthusiast ratings as a way to vet wines to try, and I wasn't paying any attention at all to the authors of the reviews, as listed in the magazine. Then a friend suggested (or more accurately, insisted) that I choose ONE particular reviewer -- and he strongly suggested Parker -- rather than randomly follow the reviews and ratings of a variety of professionals. So that's what I've been doing for the past couple of years: sticking mainly with Parker, and occasionally using other feature articles and "Best of" lists from other reviewers for a little bit of variety (though even when I do that, I still cross-check Parker's ratings for the wines).

CT members are not all professional wine tasters. But the tasting notes and scores are still worthwhile, for two reasons: (1) it's the nature of statistics that the AVERAGE scores produced from the individual reviews will ultimately be more "accurate" than any individual lay rating (PROVIDED, of course, that there are a large number of scores entered for a particular wine by a diverse group of people); and (2) the individual tasting notes, IF SUFFICIENTLY DETAILED, provide insight into why that particular reviewer scored the wine the way that he/she did.

Again, this underscores the obligation that each of us has, to provide comprehensive, detailed tasting notes and scores.

But to be "worthy" to fulfill this obligation, I still want to feel more confident and better qualified. Before I post a tasting note on CT, I want to further develop my palate, and my ability to accurately describe, in writing, my experience tasting a wine. I am planning to take a few different wine tasting courses in the next few months, with this specific goal in mind. In the meantime, I look forward to reading (and participating in) a healthy debate on the topic. Would it be better if we all followed the same structured methodology (as suggested by Corrado) in the hope that it yields more consistent -- more COMPARABLE, if you will -- numerical ratings? Or is it inevitable that there will ALWAYS be differences from one reviewer to the next, even if they follow the same approach, and those differences will ultimately be "smoothed over" by calculating a numerical average of a large number of scores?

(in reply to Corrado)

Post #: 3

RE: Standardized CT Rating system? - 1/28/2007 7:29:19 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

quote:

ORIGINAL: Serge Birbrair

I refuse to rate wines like a professional. It's their job. I don't see anybody here trying to do what dentists or surgeons do, professional wine critisism is also a job requiring certain knowledge and experience most of us don't have.

When I throw the number, it represents overall impression, something I can do,
but more and more I use binary scoring system:
1- I'd buy it again
0 - I can live without.

Works for me.

In no way am I intending any slight to Parker, Tanzer, or professional reviewers, but I disagree with the notion that wine rating and commentary should be limited to professionals, especially here at CT. While I agree that wine should not be reviewed by someone who doesn't like wine or who doesn't understand some fundamental characteristics of the varietal (for the sake of attempting to have the rating be meaningful), I don't think that MANY of the amateur winos here should squelch their voices simply because they have a day job.

The damage, as you identified, comes from reviewers who don't abide by any established guidelines and simply toss a number out there without knowing why. I'm not saying that YOU said that you do that, but I've seen enough reviews here that were "<70 -- Awful..." and "90+...EXCELLENT!!" as the SOLE review to draw into question the value of the ratings. Ratings like that are akin to submitting a Physics term paper on "The Kinesiology of Muhammad Ali's Knock-Out Punch vs. Sonny Liston" and receiving a D because the professor didn't like your penmanship (or receiving an A because he did).

I would never attempt to FORCE anyone to review a wine they didn't feel qualified to rate. CT does, however, encourage its users to review all wines marked as "Drank." I believe it would behoove us all to at least apply the same guidelines to the number that's going into that "Rating" field.

(in reply to Serge Birbrair)

Post #: 4

RE: Standardized CT Rating system? - 1/28/2007 7:55:58 PM

Serge Birbrair

Posts: 3344
Joined: 4/23/2006
From: Boca Raton, Florida
Status: offline

Corrado,
all I am saying:
why would anybody turn the process of WINE ENJOYMENT
into the hard work of Wine Rating?

_____________________________

La Tache is for amateurs.

(in reply to Corrado)

Post #: 5

RE: Standardized CT Rating system? - 1/29/2007 12:18:21 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

Corrado,
A worthwhile pursuit.
I add a few notes I've found which may add interest to the subject, firstly by the man himself:
http://www.wallstraits.com/main/viewarticle.php?id=1333
And a rather more flippant approach from Antony Rose:

"A guide to wine description
By Anthony Rose
Smell
Critics like to distinguish aroma, as in primary fruit odour like blackcurranty, from bouquet, which is the secondary, often more complex, nuances of a wine that is matured in the bottle, eg, nutty, honeyed.
Body
Body is another word for vinosity, a term that includes alcohol but also the weight and feel of the wine. Swirl a wine and the longer the "tears" that form at the rim take to descend, the greater the body. The French like to call them "legs" and the Germans call them "cathedral arch windows".
Texture
White, red and sparkling wines have different textures, a tactile quality leading to such adjectives as round, silky, smooth, creamy, fat, oily, fleshy, plump, tough, chewy, rustic and so on.
Length
Broadly speaking, the longer the elements of flavour stay with you, the higher the wine quality. One wine critic once notoriously described a wine as "long and full in the mouth, like a penis".
Balance
A quality concept describing the way that the basic constituents of wine, acidity, fruit, alcohol and tannin (in reds) come together, or jar. Is a "Dolly Parton" wine in balance? You decide."

I confess I attempt to follow the Parker 100 point approach, not because I think it's unbeatable, but because I most search for comparability. I find it easier to compare RP and WS, than RP and JR, simply because 90 vs 89-91 is easier to comprehend that 90 vs 17.5+.
I also appreciate the Parker approach in estimating future potential as I intend to lay my wines down.
I will watch further discussion with interest.
L.

_____________________________

First they came for ....................

(in reply to Serge Birbrair)

Post #: 6

RE: Standardized CT Rating system? - 1/29/2007 12:24:19 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

Oh and this article is amusing, just shows that however we standardize scoring, the differences will remain:
http://www.bipin.com/blindtasting.html

Viva La Difference!

http://www.wsmv.com/danmillersnotebook/10532937/detail.html

_____________________________

First they came for ....................

(in reply to Colonel Lawrence)

Post #: 7

RE: Standardized CT Rating system? - 1/29/2007 1:16:05 AM

ashults

Posts: 38
Joined: 9/13/2006
Status: offline

Attempts to standardize a rating system are futile at best, narrowminded at worst. Considering that the major wine critics can't agree on a single system, I would resent an attempt to impose a single system on me (not that I expect Eric would attempt such an imposition).

Adopting a single metric -- the 50 to 100 scale rather than 20 points, 4 stars, or 3 glasses -- is as standardized as cellartracker should get. How people arrive at their rating (no matter the scale) will be a combination of two factors:
1. How does a person thing his/her favorite critic(s) would rate this wine? Ultimately the critics set in people's minds what an 85 generally tastes like vs. a 90.
2. What are the person's idiosyncratic preferences about wine, e.g. fruit bomb vs. lighter elegance?

On a purely practical level, consider the number of people who think "drank at Jane's house" is a tasting note. Do you really think users will pay attention to a scoring formula if they post tasting notes like that?

(in reply to Colonel Lawrence)

Post #: 8

RE: Standardized CT Rating system? - 1/29/2007 4:55:28 AM

vaaccess

Posts: 2
Joined: 1/29/2007
Status: offline

quote:

ORIGINAL: ashults

Attempts to standardize a rating system are futile at best, narrowminded at worst. Considering that the major wine critics can't agree on a single system, I would resent an attempt to impose a single system on me (not that I expect Eric would attempt such an imposition).

Adopting a single metric -- the 50 to 100 scale rather than 20 points, 4 stars, or 3 glasses -- is as standardized as cellartracker should get. How people arrive at their rating (no matter the scale) will be a combination of two factors:
1. How does a person thing his/her favorite critic(s) would rate this wine? Ultimately the critics set in people's minds what an 85 generally tastes like vs. a 90.
2. What are the person's idiosyncratic preferences about wine, e.g. fruit bomb vs. lighter elegance?

On a purely practical level, consider the number of people who think "drank at Jane's house" is a tasting note. Do you really think users will pay attention to a scoring formula if they post tasting notes like that?

Not having a standard doesn't make sense. The fact is, cellartracker already has a "standard" by posting the rating scale that they already do. I think Corrado's point make sense. Improving on that standard to help those that want to get better, more accurate, and more consistent with their own ratings makes sense. I would think that any modification to the rating section would still allow someone to type in and rate something manually. I envision this as either a sub-form on the page or a link to a new page where they can enter in the details and those details also get recorded and displayed along with the score. The fact is, scores are indeed subjective. But, if I have more information about why someone scored a wine a 90, as in how the elements of the score were created, I can better determine whether I would like that wine, etc.

I also find it odd that people are holding back on rating wines. I bet EVERYONE that reads this thread has relied on a review on Amazon or NewEgg or some other on-line retail site. Those people aren't professionals, either, yet their ratings can be invaluable. And, often there are no professional reviews to go off of so it is the only source of information out there.

Anyway, the point is, if we are going to have a rating system, that rating system should try to be as accurate as possible. There's nothing wrong with doing that. If there is no standard or way to make a rating as accurate as possible, then the value of those ratings is lessened.

(in reply to ashults)

Post #: 9

RE: Standardized CT Rating system? - 1/29/2007 5:14:47 AM

Serge Birbrair

Posts: 3344
Joined: 4/23/2006
From: Boca Raton, Florida
Status: offline

quote:

ORIGINAL: Colonel Lawrence

Length
Broadly speaking, the longer the elements of flavour stay with you, the higher the wine quality. One wine critic once notoriously described a wine as "long and full in the mouth, like a penis".

I am afraid Playboy has copyrighted this rating system already
:(

...other than that, I am all for adopting the new CT rating system and will be the first to promote "CT peter-meter Universal wine rating scale".

Habibi, you made me laugh!
:)

_____________________________

La Tache is for amateurs.

(in reply to Colonel Lawrence)

Post #: 10

RE: Standardized CT Rating system? - 1/29/2007 5:22:13 AM

winefrog

Posts: 183
Joined: 10/23/2005
Status: offline

Interesting discussion. I don't think anything re ratings should be "required" (term from Corrado's post, though it came in conjuction with an "if" earlier). I do think that the idea of providing some guidance and/or recommendations on methodology that would be most compatible w/whatever the "standard" is would be considered helpful by many. In that light I think this would be a really good idea, but if the idea would be to restrict how contributers formulate their ratings I think it's not a good idea. There will always be wheat and chaff, and I think the emphasis ought to be on (1) helping people contribute more wheat and less chaff (guidance/guidelines/recommended practices, etc.), and (2) helping users separate the wheat from the chaff in the existing data (tag particular users to filter them out if you find their ratings and/or comments, shall I say, less than helpful). I assume tools to move blatant personal commentary from the public tasting notes to a more appropriate location (and notifying the user in question) already exist for Eric and Andrew, so some of the chaff might be dealt with at that level, at least when it comes up (not that I want/expect them to spend their time 'patrolling' these things!!).

Cheers,
Dave

(in reply to vaaccess)

Post #: 11

RE: Standardized CT Rating system? - 1/29/2007 9:59:34 AM

hutch

Posts: 54
Joined: 10/4/2006
From: Rhode Island
Status: offline

How can you standardize something that is subjective. How can you reduce wine to the sum of its parts, without taking into consideration the overall experience? As far as the ten points for ageability is concerned, shouldn't the notes convey and idea of this? I would argue that my system of arbitrarily assigning a rating is the best. And I'm right, because my ratings are most attuned with my palate. Truth be told, I only rate wines as much as I do because I'm obseessed with all the stats I can pull up with this program.
I, have however, found the tasting notes very helpful. There are many times, when researching a wine, that I can only find TN's on CT, or recent TN's on recent wines.
Also, one of the great things about CT is the difference in scores. It only points out how subjective wine really is. And I don't think the scores, especially with wines over $20, are any more divergent then if you compared them to numerous professionals.

(in reply to Corrado)

Post #: 12

RE: Standardized CT Rating system? - 1/29/2007 2:07:36 PM

billz

Posts: 138
Joined: 7/11/2006
From: Portland OR
Status: offline

Corrado - thanks for taking the time to put this information together and for broaching the topic here on the forum. I think that it's important to know what is meant when someone puts down a score of '93'. Yes, it's subjective - it must be. But it is better to have a common definition and standard practices than just a free-for-all. Having said that, I don't think it should be required that scores on CT are entered in their component parts - perhaps a configuration setting.

I have some rather basic questions on how to go about scoring a wine that I hope could be defined regardless of the level of scoring system used.

- When I rate a wine, what am I rating it against? Is it by varietal? Area? Type? Am I asking myself 'Is this the best cabernet sauvignon I've ever had?' or 'Is this the best California cab?' or 'Is this the best red Bordeaux style?' or 'Is this the best Napa cab?' or 'Is this the best red wine?' or 'Is this the best wine?'. Basically - at what level of detail am I judging the wine? I think I would give quite different scores depending on the answer.

- How do I rate a wine (varietal) that I don't care for? I don't particularly care for Syrah. But I can appreciate some of the flavors and styles and can usually differentiate a good one from a bad one.

One final comment on the Robert Parker scale. The aging potential scoring will only work for rating wines upon release. This is probably fine for Mr. Parker as that's what he does. But on CT, we're rating wines throughout their life. This is where the problem comes in. Let's say I rate a wine as a '98' - perfect except that I think it will age for 10 years (not 20+). In five years I taste the wine again and it's perfect and I think it will last 5 more years. I must rate it a '96'. This doesn't make sense.

Great discussion folks!

_____________________________

Bill

(in reply to hutch)

Post #: 13

RE: Standardized CT Rating system? - 1/29/2007 6:14:39 PM

vaaccess

Posts: 2
Joined: 1/29/2007
Status: offline

quote:

ORIGINAL: billz
Yes, it's subjective - it must be. But it is better to have a common definition and standard practices than just a free-for-all. Having said that, I don't think it should be required that scores on CT are entered in their component parts - perhaps a configuration setting.

I agree with you. It's better to at least have a standard starting ground. I also agree that the rating system should be volunteer. Someone can enter all the details if they want, but they could just type in text or a rating number, too. Those ratings with more details would likely hold more merit or should at least be more informative than one with just a number, however.

quote:

ORIGINAL: billz
One final comment on the Robert Parker scale. The aging potential scoring will only work for rating wines upon release. This is probably fine for Mr. Parker as that's what he does. But on CT, we're rating wines throughout their life. This is where the problem comes in. Let's say I rate a wine as a '98' - perfect except that I think it will age for 10 years (not 20+). In five years I taste the wine again and it's perfect and I think it will last 5 more years. I must rate it a '96'. This doesn't make sense.

I agree with you 100%. I think having an aging component hidden in the overall number is just crazy. You could have an incredible wine/varietal that doesn't age well that gets scored a 90 because it is a drink it now whereas if it could age it would be a 96 or whatever. I think the aging component of the wine should definitely be noted, but throwing it into the overall score hides the detail. Is it a good wine or is it just a decent wine that could age for 20+ years? It's easy to see how a wine could get lots of extra points just beacuse it will age, but in fact would taste worse than one that won't age as well. *IF* aging is left into the overall ranking, that's when it is vital that the drinkability window is also published and taken into consideration so someone can determine the REAL score.

(in reply to billz)

Post #: 14

RE: Standardized CT Rating system? - 1/29/2007 8:59:07 PM

ashults

Posts: 38
Joined: 9/13/2006
Status: offline

quote:

ORIGINAL: vaaccess
Not having a standard doesn't make sense. The fact is, cellartracker already has a "standard" by posting the rating scale that they already do. I think Corrado's point make sense. Improving on that standard to help those that want to get better, more accurate, and more consistent with their own ratings makes sense. I would think that any modification to the rating section would still allow someone to type in and rate something manually. I envision this as either a sub-form on the page or a link to a new page where they can enter in the details and those details also get recorded and displayed along with the score. The fact is, scores are indeed subjective. But, if I have more information about why someone scored a wine a 90, as in how the elements of the score were created, I can better determine whether I would like that wine, etc.

I also find it odd that people are holding back on rating wines. I bet EVERYONE that reads this thread has relied on a review on Amazon or NewEgg or some other on-line retail site. Those people aren't professionals, either, yet their ratings can be invaluable. And, often there are no professional reviews to go off of so it is the only source of information out there.

Anyway, the point is, if we are going to have a rating system, that rating system should try to be as accurate as possible. There's nothing wrong with doing that. If there is no standard or way to make a rating as accurate as possible, then the value of those ratings is lessened.

You seem to have lost my distinction between having a standard metric (a 50-100 scale) and a standard system such as Corrado wants. I agree that a standard metric makes sense.

However, a standard system (like Corrado's suggestion) seeks to establish for all users the "correct" relative importance of various factors for judging the overall quality of a wine. I suggest that a properly stored, newly released bottle of Chateau d'Yquem will become undrinkable in less time than it will take to reach a consensus on the "correct" importance of these factors. Consider the debate that has already started here over ageability.

Finally, let's consider your own reference to Amazon.com. Amazon has a 5-star system but does not seek to establish the relative importance for rating a fiction book of clear writing, vivid imagry, engaging plot, repeat reading potential, etc. Yet, they have produced a useful way to compare ratings because it captures each reader's overall rating of the book and gives each reader space to explain his/her rating. This is exactly the current state of cellartracker.

(in reply to vaaccess)

Post #: 15

RE: Standardized CT Rating system? - 1/30/2007 1:57:33 AM

Pontac

Posts: 450
Joined: 8/3/2006
Status: offline

Having judged at several wine competitions I can say that giving points for different aspects of the wine (color/nose/etc) just doesn't happen, the total score is given based on previous experince; indeed its more of a triage -- is the wine worthy of a gold/silver or bronze? -- Thats a 4 point scale.

And those places that do laboriously mark a few points for every aspect ofthe wine -- when they add up the points to get the total, there is often some rejigging in the various categories to get a final desired score :)

Personally I only use the Parker system on CT, and I translate from my own system using the description on CT

Extraordinary (95-100 points)
Outstanding (90-94)
Very Good to Excellent (85-89)
Good (80-84)
Average (75-79)
Below average (70-74)

And then select a score in the appropriate range.

I am also puzzled by 'Average' -- If a score of 75-79 is really the average then surely there should be a hell of a lot more wines rated at that score?

I think there is way too much weight given to scores. They are a very blunt instrument.

Then there is the other question, mentioned by previous participants in this thread. Who are we marking for?

If we are scoring wines for our benefit, thus we can base future purchases on our high scores and avoid wines with low scores, then our personal likes and dislikes are paramount. I don't like Pinot Noir and, as a personal score, will mark it low. But if I was being non-partisan, I would score it for what it is - say as a good example of the variety and the region its from - and as such give it a much higher score even though I personally do not like the wine.

I think that, in the end, I use CT as my database of wines that I own and have tasted, and as such the scores reflect my personal tastes. If others want to view my TNs they are welcome to. But I don't really care what people who I don't know think of wines.

< Message edited by Pontac -- 1/30/2007 2:37:47 AM >

(in reply to Corrado)

Post #: 16

RE: Standardized CT Rating system? - 1/30/2007 3:00:44 AM

Serge Birbrair

Posts: 3344
Joined: 4/23/2006
From: Boca Raton, Florida
Status: offline

quote:

ORIGINAL: Pontac

I think that, in the end, I use CT as my database of wines that I own and have tasted, and as such the scores reflect my personal tastes. If others want to view my TNs they are welcome to. But I don't really care what people who I don't know think of wines.

you got it!!!!!

But being a good sport,
I'll be more than delighted using any standard system adopted as soon as Jancis, Parker and Clive Coats agree on Pavie score, or come within 3 points from each other,
from 2000 and up.

_____________________________

La Tache is for amateurs.

(in reply to Pontac)

Post #: 17

RE: Standardized CT Rating system? - 1/30/2007 5:44:08 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

Corrado et. al.
You may enjoy this 'attack' on numbers:
http://www.finewinemag.com/docs/Scoring%20Wines%20by%20Atkin.pdf

_____________________________

First they came for ....................

(in reply to Serge Birbrair)

Post #: 18

RE: Standardized CT Rating system? - 1/30/2007 10:14:39 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

quote:

ORIGINAL: Pontac

Having judged at several wine competitions I can say that giving points for different aspects of the wine (color/nose/etc) just doesn't happen, the total score is given based on previous experince; indeed its more of a triage -- is the wine worthy of a gold/silver or bronze? -- Thats a 4 point scale.

And those places that do laboriously mark a few points for every aspect ofthe wine -- when they add up the points to get the total, there is often some rejigging in the various categories to get a final desired score :)

Personally I only use the Parker system on CT, and I translate from my own system using the description on CT

Extraordinary (95-100 points)
Outstanding (90-94)
Very Good to Excellent (85-89)
Good (80-84)
Average (75-79)
Below average (70-74)

And then select a score in the appropriate range.

I am also puzzled by 'Average' -- If a score of 75-79 is really the average then surely there should be a hell of a lot more wines rated at that score?

I think there is way too much weight given to scores. They are a very blunt instrument.

Then there is the other question, mentioned by previous participants in this thread. Who are we marking for?

If we are scoring wines for our benefit, thus we can base future purchases on our high scores and avoid wines with low scores, then our personal likes and dislikes are paramount. I don't like Pinot Noir and, as a personal score, will mark it low. But if I was being non-partisan, I would score it for what it is - say as a good example of the variety and the region its from - and as such give it a much higher score even though I personally do not like the wine.

I think that, in the end, I use CT as my database of wines that I own and have tasted, and as such the scores reflect my personal tastes. If others want to view my TNs they are welcome to. But I don't really care what people who I don't know think of wines.

Pontac, lest you think I'm singling out your post from the others, let me say that you hit a lot of the points I was trying to address. I've purposefully avoided this thread for a day to let the discussion go and not stick my nose in and reply to every single post (that will come later when I 'bump' the thread.

).

I do want to be clear that I'm not at all trying to 'force' a system on anyone or be a pretentious wine snob. I've seen a few mentions to wine ratings being subjective (can't argue there) and to folks prefering to stick with Parker's method.

I can't agree more. I would have elaborated more in my first post, but it was getting pretty long (and probably boring) as it was.

First, re: subjectivity. Hell yes! Part of the problem is different tastes, something we'll have to live with. Another is fundamentally different systems of rating a wine. I 100% agree that any component-based system should be optional, but I also would hope that, if such a system were adopted, non-component ratings could be filtered out at a user's choosing. Personally, I've found that I can be more objective in my ratings if break it down as opposed to basically giving a wine the gladitorial thumbs up (90) or thumbs down (70).

In consideration of Robert Parker, I chose his model BECAUSE CT directly references his methodology. I got a chuckle out of the suggestion that folks should just 'rate wine like Parker' and not mess with the component crap.

From the page CT links to for ratings: "In terms of awarding points, my scoring system gives every wine a base of 50 points. The wine's general color and appearance merit up to 5 points. Since most wines today are well made, thanks to modern technology and the increased use of professional oenologists, they tend to receive at least 4, often 5 points. The aroma and bouquet merit up to 15 points, depending on the intensity level and dimension of the aroma and bouquet as well as the cleanliness of the wine. The flavor and finish merit up to 20 points, and again, intensity of flavor, balance, cleanliness, and depth and length on the palate are all important considerations when giving out points. Finally, the overall quality level or potential for further evolution and improvement—aging—merits up to 10 points." source: http://www.erobertparker.com/info/legend.asp

My thought is that if CT is pointing users to that site as a basis for ratings, it indicates some desire to form some sort of standard within the CT community. If there are going to be Community reviews for CT, how can there NOT be some form of rubric applied to at least shepherd users into considering the same elements when punching in two digits on their keyboard before clicking "CONSUME BOTTLE."

Regarding the question about what the rating should be against (Do I like it? vs. Is it a good Red Wine? vs. Is this the best Red Wine I've ever Had? vs. Is this the best Cabernet I've ever had vs. How does this rate against a 'perfect' Cabernet? vs. .... 'perfect' California Cabernet? vs. ... 'perfect' Napa Valley Cabernet?) I don't know. I haven't had a lot of 'GREAT' wines (using great as a synonym for expensive), but I have enough of an idea of the potential of Cab or a Pinot or a Shiraz to be able to taste a lesser one and reasonably identify faults. By providing something of a guideline for ratings, CT would not only improve their own ratings, but it could potentially serve as an educational tool for those looking to learn more about wine and what some reviewers might look for in a 'good' wine.

More later. I've already rambled for 2 paragraphs more than I intended.

(in reply to Pontac)

Post #: 19

RE: Standardized CT Rating system? - 2/2/2007 8:19:39 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

quote:

ORIGINAL: vaaccess

quote:

ORIGINAL: billz
One final comment on the Robert Parker scale. The aging potential scoring will only work for rating wines upon release. This is probably fine for Mr. Parker as that's what he does. But on CT, we're rating wines throughout their life. This is where the problem comes in. Let's say I rate a wine as a '98' - perfect except that I think it will age for 10 years (not 20+). In five years I taste the wine again and it's perfect and I think it will last 5 more years. I must rate it a '96'. This doesn't make sense.

I agree with you 100%. I think having an aging component hidden in the overall number is just crazy. You could have an incredible wine/varietal that doesn't age well that gets scored a 90 because it is a drink it now whereas if it could age it would be a 96 or whatever. I think the aging component of the wine should definitely be noted, but throwing it into the overall score hides the detail. Is it a good wine or is it just a decent wine that could age for 20+ years? It's easy to see how a wine could get lots of extra points just beacuse it will age, but in fact would taste worse than one that won't age as well. *IF* aging is left into the overall ranking, that's when it is vital that the drinkability window is also published and taken into consideration so someone can determine the REAL score.

As you both point out, this is a troublesome issue with Parker's rating system; he's assigning points based on speculation. I think I could effectively argue either side of this option so long as the reviewers are consistent.

To have any real meaning, it's absolutely true that a 100 point wine, tasted from the womb, may rate no better than 91 or 92 at its peak (assuming you consider 1 or 2 being 'wine has peaked'). If the reviews are consistent in how they determine a score, a rating may actually tell something more about the wine than a simple letter grade score (e.g. B+ effort). Any wine rating in the 90s MUST have some cellarability potential and ANY wine having a 95+ score should be cellarable for some time (given proper care).

On the flip side, how would a rating devoid of age (or any structured scoring methodology) have value and meaning? What does a 95 score tell you other than, "Dude(ette) liked that wine!" Like it or not, aging potential is a factor in purchase decisions. I recognize that CT is not set up to be research site, but in reading other threads it's evident that folks do look at how others have reviewed wines and use the community data here to consider purchases. If an adopted rating system is known to consider this, it should NOT be distubing to see a wine's score fluctuate and, in the case of mid to high 90s wine, see its peak rating early in life.

Again, all this assumes standardization and consistency. I wouldn't be heartbroken to see the 10 pts given to "aging potential" get reallocated to an 'overall impression' catgory that could alternatively describe the quality of the product (into which age could be considered) and how the reviewer believed the components of the wine come together to make the final product.

I still have some more ranting to do, but I'll leave that for later (and go have a glass of scotch).

(in reply to vaaccess)

Post #: 20

RE: Standardized CT Rating system? - 2/2/2007 8:52:56 PM

winefrog

Posts: 183
Joined: 10/23/2005
Status: offline

Corrado (et al.),

You know, this "embedding" of aging potential within a generalized score is a nit I hadn't thought to pick at, but it is an important one once you step outside of the Parker mode of (mostly) reviewing wines before and (relatively) shortly after release. Even within that mode I guess I hadn't put two and two together on this. Hmm. So (as an aside) I guess it might be worth while paying attention to relatively highly rated wines with short drinking windows, as that would suggest that the wine is in some sense "better" now than an equally rated wine with a longer drinking window. I'm assuming here that the drinking window (at least the far end of it) correlates directly to whatever WA folks are considering when they give the 0-10 points for aging potential.

In any case, I fully agree that these 10 points, if the "methode WA" were applied, would be a really tricky thing outside of their usual peri-availability rating mode.

What to do? "overall impression?" Hmm... Certainly a worthy question...

Thanks for continuing to shepherd this thread along, rather than letting it die the quiet death that most threads on most sites quickly succumb to. It's clear you've thought about this quite a bit.

And now, for a Very Dry Martini...

Cheers, and thanks,
Dave

(in reply to Corrado)

Post #: 21

RE: Standardized CT Rating system? - 2/2/2007 11:20:00 PM

hutch

Posts: 54
Joined: 10/4/2006
From: Rhode Island
Status: offline

Corrado said:

As you both point out, this is a troublesome issue with Parker's rating system; he's assigning points based on speculation. I think I could effectively argue either side of this option so long as the reviewers are consistent.

To have any real meaning, it's absolutely true that a 100 point wine, tasted from the womb, may rate no better than 91 or 92 at its peak (assuming you consider 1 or 2 being 'wine has peaked'). If the reviews are consistent in how they determine a score, a rating may actually tell something more about the wine than a simple letter grade score (e.g. B+ effort). Any wine rating in the 90s MUST have some cellarability potential and ANY wine having a 95+ score should be cellarable for some time (given proper care).

On the flip side, how would a rating devoid of age (or any structured scoring methodology) have value and meaning? What does a 95 score tell you other than, "Dude(ette) liked that wine!" Like it or not, aging potential is a factor in purchase decisions.

Yeah, pretty much. Which is pretty much what any score tells you... The ability of a wine to age means different things to different people, just like all aspects of wine. Some people want to buy 5 cases of one wine and drink it through its progression. Others want a wine that tastes good right now... While the ability of a wine to change and metamorphosize over years is a wonderous thing, it doesn't a whole lot if you went out and bought one bottle, popped it, and are drinking it right now... What do those ten points mean for that person? Right now, I'm enjoying a great bottle of Soos Creek. But it's a little oaky and probably needs a couple of years. But while I think that, I have no idea. All I really know is that it has great fruit and stuffing, but is a little woody for me right now... All of these thoughts can not be conveyed in a score... So why such an interest in standardized ratings. What the taster says is much more important and enlightining anyway. Every time I use CT, I skim over the TN's, b/c that tells me what I want to know. Just like if I use Amazon I don't look at how many stars a product averages. I skim the reviews. But nice post. Nice to see all the responses. Obviously an issue a lot of people have opinions on...

< Message edited by hutch -- 2/2/2007 11:34:24 PM >

(in reply to winefrog)

Post #: 22

RE: Standardized CT Rating system? - 2/4/2007 12:37:12 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

"Corrado said:
As you both point out, this is a troublesome issue with Parker's rating system; he's assigning points based on speculation. I think I could effectively argue either side of this option so long as the reviewers are consistent. "
Aye, there's the rub....... http://www.davidpbrown.co.uk/poetry/william-shakespeare-3.html

What would you expect when the same blind tasting organisation drank the 1st Growths of Bordeaux on two separate ocassions?
Of course - exactly opposite rankings! Correlation -1.0!
http://www.erobertparker.com/EWS/ews13.asp

http://www.erobertparker.com/EWS/ews71.asp
My favorite example of inconsistency.
And no I don't believe the wines changed that much between tastings.

_____________________________

First they came for ....................

(in reply to hutch)

Post #: 23

RE: Standardized CT Rating system? - 2/4/2007 9:07:04 AM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

quote:

ORIGINAL: Colonel Lawrence

"Corrado said:
As you both point out, this is a troublesome issue with Parker's rating system; he's assigning points based on speculation. I think I could effectively argue either side of this option so long as the reviewers are consistent. "
Aye, there's the rub....... http://www.davidpbrown.co.uk/poetry/william-shakespeare-3.html

What would you expect when the same blind tasting organisation drank the 1st Growths of Bordeaux on two separate ocassions?
Of course - exactly opposite rankings! Correlation -1.0!
http://www.erobertparker.com/EWS/ews13.asp

http://www.erobertparker.com/EWS/ews71.asp
My favorite example of inconsistency.
And no I don't believe the wines changed that much between tastings.

Colonel-

I'm confused by your assessment of 'exactly opposite rankings' and 'inconsistency' when there appears to be a direct 1:1 correlation (tossing 'ties' aside) in the rankings. I've added the original scores (Column = Score) to the Parker spreadsheet for each wine tasted & scored in the second session (Colum = Total). I can only assume you considered the descriptions on those pages to be 1-Nth place as scored, not the wines 1-N (as bagged).

To me, that's RESOUNDING PROOF that 50-100 independant reviewers can blindly score a number of wines very consistently over time. One of us is CLEARLY missing something (and I certainly understand it could be me), so I'm hoping you'll check back, review the links & data, and clarify to me where the inconisistency is you're alluding to.

This is good stuff! I really appreciate all the feedback!

(in reply to Colonel Lawrence)

Post #: 24

RE: Standardized CT Rating system? - 2/5/2007 3:11:53 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

Erin,
Did you get confused by the years?
This is what I meant:
MR 1st in one list, last in the other.
HB 2nd .................4th..............
Latour 3rd..............3rd............
Margaux 4th...........2nd...........
Lafite 5th ................1st...............

That's pretty opposite to me.
I do all my tastings by paired comparison as I find it more rewarding.
L.

(in reply to Corrado)

Post #: 25

RE: Standardized CT Rating system? - 2/5/2007 4:32:18 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

Ah, I gotcha now. I thought you were trying to point out two separate events. It wasn't clear that I was supposed to pair wines across the two tastings.

What do you make of the fact that 50-100 folks formed the basis for these reports and came to such varying conclusions? I could certainly see a handful of tasters getting palate du jour, but there was evidently some factor that caused the variance (notes on the latter tasting suggest a much higher than normal number of bad bottles).

Maybe if they had a more standardized scoring system the results would have been more repeatable.

< Message edited by Corrado -- 2/5/2007 4:33:59 PM >

(in reply to Colonel Lawrence)

Post #: 26

RE: Standardized CT Rating system? - 2/6/2007 11:55:46 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

I was surprised by these results: the organisation conducting them was the same and they have the RP reputation to protect.
I can only come to the conclusion that there is more variability in tasters than in the bottle of wine.
This additional research gives a hint as to the extent of the variability:
http://www.bipin.com/blindtasting.html
Rgds,
L.

(in reply to Corrado)

Post #: 27

RE: Standardized CT Rating system? - 2/7/2007 9:07:03 PM

Corrado

Posts: 110
Joined: 1/19/2007
From: Vermont
Status: offline

quote:

ORIGINAL: Colonel Lawrence

I was surprised by these results: the organisation conducting them was the same and they have the RP reputation to protect.
I can only come to the conclusion that there is more variability in tasters than in the bottle of wine.
This additional research gives a hint as to the extent of the variability:
http://www.bipin.com/blindtasting.html
Rgds,
L.

One person being at odds with a group is expected, but the likelihood of two independant groups of experts (with some likely overlap between the groups of 50 & 99 tasters) being so profoundly different is, from a research standpoint, unlikely without there being other influencing factors.

Getting back onto topic, I'd like to continue to flesh out a CT-suggested rating. As the Colonel has shown, events where wines are only scored relative to their peers can lead to subjective evaluations where a wine is compared directly to other peers more than being evaluated on its own merits. Anyone who watchs WineLibraryTV can watch Gary Vaynerchuk almost admittedly score down a wine after he tastes an 'awesome' wine.

Again, my goals aren't to force a hard & fast system on users or tell anyone that they should score wine like I do. A more standardized system cannot eliminate the day-to-day variance of a particular person's palate nor the person-to-person preferences for a particular flavor profile or style of wine. What it can do is ensure that both reviewers apply similar scrutiny to their ratings so that any rating variations that appear on CT.com are most closely linked to taster preference than simply scoring variation.

Level the playing field. Let some folks play forward, others midfield or defense, and the eccentric ones can play keeper, but they're all playing the same game with the same rules. A football/soccer game with some folks trying to play American football with others trying to play FOOTBALL and still others playing rugby or Ultimate Frisbee end up with a meaningless, pointless game.

(in reply to Colonel Lawrence)

Post #: 28

RE: Standardized CT Rating system? - 2/7/2007 9:34:01 PM

hutch

Posts: 54
Joined: 10/4/2006
From: Rhode Island
Status: offline

Unless, of course, the very act of scoring a wine is a pointless, meaningless game...

(in reply to Corrado)

Post #: 29

RE: Standardized CT Rating system? - 2/8/2007 1:10:40 AM

Colonel Lawrence

Posts: 4339
Joined: 4/24/2006
From: Arabia
Status: online

The interesting thing about the one taster being at variance to the mass was that it may indicate either:
1. Big differences in taste by geography (part of the Old vs New World debate).
or
2. Differences through 'educated' vs uneducated palate

It seems that like many things (smoking, drinking beer etc.) drinking good wine (especially complex stuff) may be an acquired taste/habit. Tastes certainly change - I used to add 3 spoons of sugar to my tea, and now can't bear any; dark chocolate seems an adult occupation, verses milk chocolate for kids; different styles of cereal (sugar smacks v meusli); etc..
Don't think I've added much to the Standardizing debate, but personally it's the complications (complexities) with wine that I like - viewed in a scientific sort of way of course ;)
L.

PS If the above thoughts have their basis in fact it means that scoring must take full account of who the scorers are. Unless of course mass scoring eliminates, in a satisfactory way, the differences. I do fear that average scores may be just that though - average. This is why I study multiple sources for scores (e.g. EWS, CT) and individual opinions (e.g. RP, WS, JR, IWC ...) - especially my own!

_____________________________

First they came for ....................

(in reply to hutch)

Post #: 30