Critiquing Chocolate Judging | #PodSaveChocolate Ep 143

When and Where to Watch

This LINKEDIN link is shareable.

Watch and comment LIVE or view the archived episode on LinkedIn. Join my network on LinkedIn to receive notifications and to refer business to each other.

Watch this episode on LinkedIn

This FACEBOOK link is shareable

Watch and comment LIVE or view the archived episode on TheChocolateLife page on Facebook (for 30 days, then watch the archive on YouTube).

Follow TheChocolateLife on Facebook to receive notifications and catch up on other content.

Watch this episode on Facebook

Episode 143 Overview

One of the first things a chocolate critic needs to have is a rubric for evaluating what they will be critiquing. It turns out that developing these rubrics is not necessarily straightforward, with a lot depending on who the audience is and what you want to communicate.

Definition: A rubric is typically an evaluation tool or set of guidelines used to promote the consistent application of expectations, objectives, or  standards.

A case in point: Arriving at my first rating system took me around four years; I first started seriously looking into them in 2007 and did not launch chocophile.com (June 2001) until I had one I felt comfortable with and reflected that I was judging chocolate, which is very different from wine, beer, and other beverages and foods.

Judging Innovation in Cocoa | #PodSaveChocolate Ep 142

Episode 142 of #PodSaveChocolate features an exploration of judging innovation in cocoa, from the perspective of a judge (me) of the 2025 Ecuadorian Cacao Innovation Award, with a BONUS tasting! [Updated]

The Chocolate LifeClay Gordon

Early on, I explored numerical rating systems like the ones used by Wine Spectator and other wine publications. These tended to use 1-100 scales, presenting a single number that was the composite of all of the impressions of the reviewer. Different critics from different publications might give different ratings for the same wine: the number that truly mattered was the one Robert Parker gave.

In the end, I decided not to go with a numerical rating system, though everywhere I looked, there they were looking back at me.

My Critique of Numerical Rating Systems

1. They are devoid of personality. The results are just a number; they don’t tell you anything about the reviewer or the process of arriving at the final score.
2. They are abstract. On a scale of 1-10, what does a 7 mean?
3. What’s better? A 1-10 scale or a 1-100 scale?
4. They require calculators. If there are multiple criteria (e.g., aroma, taste, mouthfeel, etc), they are not equally considered, i.e., taste might count for 40% of the final score and aroma 10%.
5. The numbers have inherent bias. The difference between 87 and 89 is two points. The difference between 89 and 91 is two points. However, 91 starts with a 9, which imparts a psychological bias.
6. When judging in panels, precision (the number of places to the right of the decimal point) becomes an issue. A) Can any person reliably judge the difference in taste to one part in a hundred?; and B) Differences as small as 1/100th of a point can mean that an entry might drop to a lower awards category or not win an award at all.

Critiquing Personally vs Judging Panels

For this discussion, it’s really important to understand that developing a rubric for personal use, that is, on your own and not in conjunction with others, is different from developing a rubric that two or more people are going to be using.

One approach may not be better than another in general, but some approaches may be better in specific circumstances.

Whatever the rubric is, the elements must be defined and communicated as unambiguously as possible so that those who are reading your reviews can understand what you are evaluating and how.

One Golden Rule

The OG Chocophile Rating System (2001)

My OG rating system, developed for chocophile.com, differs from numerical systems because it focused on communicating perceived value: Given the price and style of a chocolate, was the product I was tasting “worth it”?

• The Price Range established one set of expectations. My expectations for a Mass-Market product (at the time, costing less than US$15/retail per pound / ~$33/kg) are lower than for a product in the Prestige price tier (at the time, greater than $45/lb / ~$99/kg).
• The Style gave a clue about the overall flavor profile. American-style products (no matter where they were made) tend to be sweet with flavors that outshine the chocolate.
• The Ranking is a short description of value: For this price and that style, the product is “Ordinary,” for that price and this style, the product is “Extraordinary.”
• I chose the word “ordinary” for the midpoint as it carries with it more nuanced connotations than “average.”

Critiquing the Academy of Chocolate Judging

Aoc Judging System

This page contains only a very high level overview of the process.

The Judging Process

Critiquing the International Chocolate Awards Judging

ICA Judging System

The judging system used by the ICA is explained on this page. The ENGLISH form download links DO NOT correspond to the full judging rubric presented, rather to the “triage” round, the Italian versions of the forms are more reflective of the judging process.

Judging System and Forms

Separately, there is space for feedback.

Critiquing the Good Food Awards Judging

The Good Food Awards does not publish its judging rubrics or official judging standards. In 2015, when I was judging the Confections, there was so standardized set of guidelines.

There are Rules and Regulations around how to enter and what can be entered, but nothing substantive on how judging is actually performed (there are no judging rubrics, just principles).

15 Years of the Good Food Awards | #PodSaveChocolate Ep 134

Episode 134 of PodSaveChocolate explores the 15-year history of the Good Food Awards in the Chocolate and Confections categories. Insights into the results, as well as into what it took to compile and clean the data. (TL;DR – extracting and cleaning the data was NOT pretty.)

The Chocolate LifeClay Gordon

My Rubric for the 2015 Confections Judging

My approach as the “table head judge” (there were multiple tables, each with its own “head” judge) was to ask each judge at my table to rank order their top five picks for each flight (there were multiple tasting flights of 7 to 9 entries).

I did not give any specific criteria for how to rank order the picks. I then polled each judge for their top five picks, in order, creating a table of picks. The picks were tallied, and the entries with the most first, second, and third-place picks went through.

Critiquing the Chocolate Alliance/NW Chocolate Festival Judging

The website is out of date and contains zero information about the judging process.

Awards | The Northwest Chocolate Festival

The Northwest Chocolate FestivalDEI Creative in Seattle, WA

The Discover Chocolate/ChocolateLife Emotional Rating System

I often add to my Zero rating the comment “... and I would not regift the chocolate to anyone whose opinion matters to me.” What this does is let a reader know what I really think about the chocolate I am rating.

The fun part of this approach is that everyone gets to write their own descriptions for each of the levels, communicating how they feel using their own words.

Observations about the Ecuadorian Cacao Innovation Awards (2025)

A Modern Approach – Tier Lists

These days, tier lists are everywhere, but they became popular in video game communities. They are an adaptation of A-F grading systems (adding an “S” tier that is higher than “A”).

Where tier lists fall short is that S, A, B, C, D, F is six tiers; their is no midpoint with the same number of levels above and below. What I have done is to add a tier between “C” and “D” ⋯ “O” (the letter o, for ordinary).

You’ll notice that this is not unlike the seven-step ranking in the original chocophile rating system, with expanded definitions that don’t delve into the emotional connection.

What makes this rubric useful is that today, tier lists are meme-able; tools for ranking just about anything, and are widely shared on social media channels, with dedicated tools to create and distribute them. Tier lists serve to help communities discuss preferences in structured ways, reflecting how collective opinions evolve as a topic is updated or as new information emerges.

Questions?

If you have questions or want to comment, you can do so during the episode or, if you are a ChocolateLife member, you can add them in the Comments below at any time.

Episode Hashtags and Socials

#TierLists #RatingChocolate #ReviewingChocolate
#chocolate #chocolat #specialtychocolate #craftchocolate #beantobarchocolate
#cocoa #cacao #cacau
#PodSaveChoc #PSC
#LaVidaCocoa #TheChocolateLife

Future Episodes

#PodSaveChocolate and #TheChocolateLifeLIVE Archives

To read an archived post and find the links to watch archived episodes, click on one of the bookmark cards, below.

Pod Save Chocolate Calendar and Archive

News, views, and conversations on topics in cocoa and chocolate streamed live to YouTube, LinkedIn, and Facebook. #PodSaveChocolate!

The Chocolate LifeClay Gordon

#TheChocolateLifeLIVE Archive

News, views, and conversations on topics in cocoa and chocolate streamed live to YouTube, LinkedIn, and Facebook.

The Chocolate LifeClay Gordon

PodSaveChocolate

Hosted by Clay Gordon, the creator and moderator of TheChocolateLife.com, #PodSaveChocolate covers a wide variety of topics in the worlds of cocoa and chocolate. The video versions of this podcast are hosted and archived on YouTube, LinkedIn, and Facebook. Click on the PodSaveChocolate link in the top nav on TheChocolateLife for the links to the post accompanying each episode.

Pinecast

Become a Premium ChocolateLife Member!