Judging Competition BBQ – Part 1 by Donna Fong, Butchers Daughter BBQ

April 17th, 201418 Comments

 

Pitmaster facing six judges in a SYD backyard contest

Pitmaster facing six judges in a SYD backyard contest –  courtesy of Brenda Magee Photography

 

<Part 2><Part 3>

The most common question I hear when I tell people I am a certified BBQ judge is How do you become a judge?”  That’s the easy question to answer. I admit that it seems like a pretty good racket. You take a BBQ judging class from one of many sanctioning bodies that exist. I’m certified to judge BBQ by two organizations, the Kansas City BBQ Society and the Pacific Northwest BBQ Association. To be truthful, you don’t have to be certified to judge competition BBQ but being so makes it easier to get into a contest.

The second question I get is a little harder to answer and the nuances of the answer lies in better kept secrets than those made in Las Vegas. The question is “How do you judge BBQ? It seems harmless enough of a question. I usually say that I am required to judge according to appearance, taste and tenderness and the scores are weighted with taste having the most weighting and appearance having the least weighting.

Handpicked judges for the 2012 Kingsford Invitational in Belle, Missouri, to judge the battle of the 8 World Champions

Handpicked judges for the 2012 Kingsford Invitational in Belle, Missouri, to judge the battle of the 8 World Champions

 

For KCBS, the scoring scale is from 2 through 9. The appearance / taste / tenderness criteria makes up 14%, 57%, and 29% of a meat score. For PNWBA, the scale runs from 2 through 10, and it makes up 19%, 51% and 30%. Those are the numbers but beyond the numbers is the more subjective question of how do you judge a bad, average, or excellent submission.  What does the meat have to look like, taste like, and feel like to be judged as excellent? How do you mentally compare a perfect score with something you have personally experienced?

Ben Lobenstein, KCBS Master Judge and California contest promoter says that if you can recall the best steak you ever ate in your life, that steak should be given a score of 11 on a scale of 2-9. If you remember the entry after leaving the contest, it is a 9. If you remember it for a long time, it is a 10. Others have described an average score (6) as BBQ you’d eat in a restaurant.

Harry and I recently competed in Sam’s Club National BBQ tour in Sacramento, CA. The top 6 teams overall out of 30 teams gets to move on to the regional contest in Las Vegas. There, Harry finished 3rd and I finished 11th. Troy Black, All-Star Pitmaster and Sam’s Club National BBQ Tour Ambassador, commented at the awards ceremony that the judges had given the cooks the lowest score of any Grand Champion in the series. I checked and he’s right. Big B’s Down-N-Dirty scored 662.8116 out of a possible perfect 720. Grand Champions so far this year have scored between 670 and a blistering 694. Mr. Black said that the BBQ was just as good in Northern California as it was anywhere else and he wasn’t sure why the scores were so low. The local judges were given the dubious honor of being the lowest scorers not only in California but possibly the entire country. Can we gather any clues from the scores to determine if the BBQ was judged fairly?

20130511_LongBeachJudges

Judges are lined up from most senior to least senior before being paired junior-w-senior during table assignments

 

With the new score system put in place by KCBS, teams can now see how their entries do against others at the table and how the very same judges and tables judge other meats from other teams. Teams have developed terms like the “golden-table” or the “table-from-hell” based on this transparent disclosure on how they were scored. But is it really a table of BBQ hating judges or did the table just have the luck of receiving a lot of really great or really bad BBQ? I don’t think you can reasonably argue either case. Even the best teams can have an off day of cooking. Like most teams, I’ve been disappointed on those days when I thought I cooked well enough to deserve a walk but didn’t. And, I’ve also had days when I didn’t think I cooked well but got a walk anyways.

I do know a few things about judging BBQ at a table of judges. Many years ago, I sat with a table full of experienced judges and ate BBQ entries from a single experienced cook. It surprised me that we were all within a point or two of each other’s scoring, despite our personal preferences. I was proud because I sat next to one of the best judges around. We all felt similar about the entries and scored it up or down based on how well the BBQ was done.

What also surprised me was that many years later; I judged at the American Royal in 2010 and found my scores to be consistently lower than my counterparts. For example, if there was a dry brisket entry, I’d score it a 7 and my counterpart scored it an 8 or 9 for tenderness. We both thought the same of it but what a dry brisket translated to on a score sheet was very different. All I could say was Wow.

Do I as a judge, have higher standards than some others? Does living in the culinary Mecca of wine and food have some influence on how I score? Does the fact that I have tasted flavors from some of the most celebrated chefs in the world and have traveled and eaten a variety of foods from the other side of the planet influence how I judge? Maybe. However, I do know I give out plenty of 9’s and there’s rarely a contest I don’t give a perfect 9-9-9 a few times.

As a cook, I realize how hard it is to translate what I CAN cook to what I DO cook outside of my home, in some parking lot hours away. None of it is easy. That insight will manage my expectations as judge. I know none of the cooks have the luxury of creating what Thomas Keller can create at the French Laundry. There are no rules for Keller to follow. He creates his own rules. But cooks have rules and budgets to manage. The competition circuit has numerous environmental factors and the stress

At the same time, when I enter a tent I have expectations of myself, and of my colleagues. I want to have a good time and I do. But judging BBQ is serious business and I try to bring all that I know about appearance, taste and tenderness to table, judge fairly, and be a little lenient. If I can’t decide between a 7 or an 8 and it takes me more than a few seconds to think about it, I’ll go higher. The other thing I try to do is share what I know. Judges should always talk about how and why they scored the way they did and learn from each other. With as many judges as we have in California, more experienced judges should transfer the art of judging to others and consider that process as important as judging itself.

Destination America's 2012 Kingsford Invitational to determine the best of the 8 reigning World Champions

TV’s Destination America 2012 Kingsford Invitational to determine the best of the 8 reigning World Champions

 

The new KCBS scoring system gives more transparency to cooks but it can also turn bad results into witch hunts. Judges get all of the blame and none of the credit. And that’s too bad. I’ve turned in BBQ so riddled with mistakes that I should have wrote an apology letter and taped it to the top of the box.  “Sorry you had to eat this.” or “I overslept and forgot about this meat.”  No matter what, the judge still has to eat it. No one ever says thank you for eating my bad meat. But a single judge giving a low score at a table is questioned, even if that score is thrown out. It’s too bad that while it acceptable to have a learning curve for cooks, a learning curve for judges is fodder for fiery Internet forum threads. And who is to say the low score comes from the new judges anyways? I’ve seen one of our senior most experienced judges give a single 9-8-9 to an otherwise perfect 180 entry.  His score was dropped but if five out of six judges called it 9-9-9, what could possibly be wrong?

Cooking is an art and judging is an art. If you have the fortune of being a part of competition BBQ, be thankful for the masterpiece, forgive the faults and hope for the best. It is all part of the ride.

Read <Part 2><Part 3>

 

Getting honest feedback is part of the game

Getting honest feedback during a backyard contest is part of the game – pic courtesy of Brenda Magee Photography

18 comments... read them below or add one

  1. Robert Charpentier says:

    Thank intersting read

  2. Anna Paul says:

    This is a very interesting article. I really do have to go to one of these classes someday soon. Firstly because I SO LOVE anything that has been smoked low & slow, it has such a depth of flavor nothing else can compare to it. I throughly enjoy going to all the BBQ competitions down here around the San Diego area & tasting all the various Q at them. And secondly because I have been a professional taste tester for the Senomyx Corp. www.senomyx.com for the last 9 years. Through us taste testers they conceive & develop new flavor enhancers for the major food Corporations. I can tell between less than 3% difference in salt, sugar, sour, bitter & umami. Thank you so much for sharing this. I am so looking forward to the next part or parts of this article.

    • Donna Fong says:

      Thanks Anna. Yes indeed. Many of our judges can taste elements that the cooks themselves don’t notice. I was once told that a cook heard a few times that there was a chemical taste to his BBQ. He finally discovered that a gasket he was using on his smoker was not compatible with high heat. Once he changed it, the comments went away. Wow! Hope to see you in class.
      Donna

  3. Anna Paul says:

    One more thing the reason why I love Q so much is that BAM, it totally hits all the flavor components of our tongue, sweet, salty, sour, bitter & umami at the same time, so they (our taste buds) are ALL turned on at same time.That, to me, as a super taster is like a foodie HEAVEN.

    • Harry Soo says:

      Hey Anna:
      Welcome to my website and to the articles relating to smoke and fire. I can tell you that many of my CBJ (certified BBQ judge) friends do have a keen sense of taste.
      For instance, they can detect the nuance of whether someone sprayed their meat with apple juice or pineapple juice.
      I’ve found I’ve scored better spraying with just plain water so as not to leave any trace of fruit and let my barbecue stand based on the meat flavor.
      BBQ judging is a one bite affair as a judge tastes about 2 oz X 6 samples each of chicken, ribs, pork, and brisket for about 48 oz per contest.
      So we try to wow them with an explosion of flavor that’s a symphony of flavors in their mouth. So comp barbecue is designed for a one bite experience. When I cook for family and friends, I have to tone down the saltiness and flavors a bit.

  4. Bill Moth says:

    Great article.Having judged as a CBJ for 10 years,traveled thousands of miles.I have meet some of the best people(judges and teams).

    • Donna Fong says:

      I agree with you 100% Bill!

    • Harry Soo says:

      I will find out Bill, in 2014 and beyond, as I’m staring the judging phase of my barbecue career leading to a Master CBJ (after I’ve judged 30 KCBS contests). I’ve judged a handful of times but they don’t count towards the MCBJ and they were KCBS Invitationals and backyard contests. So I’ll be at the kids table next month . . . as the most junior judge. Be kind to me! :-)

  5. I like Ben’s criteria for scoring. I have also heard from an East Coast judge that you only score 9’s if you were at a family gathering you would hide the ribs and eat them yourself. Both are humorous but when do you score a nine? I left the certification class with only an inkling of how to score. My rookie contests I scored lower than I do now. I bet if you ask any judge when to score a 9 you will receive numerous answers.
    How can scoring be more consistent? Maybe the required cooking with a teams should be mandatory before judging? Defining the point system should be addressed. What actually does scoring a 9 really mean.What does average really mean? I too have seen experienced judges scoring low. I also usually have a few inexperienced and sometimes non certified judges at my table. The system is not perfect and is evolving.I feel privileged and honored to be a judge. I may have a couple of contests under my belt but I still feel like this is a constant learning experience.

    • Donna Fong says:

      I agree with you Chilebrown! I think that the moment that I stop learning, I will lose interest in judging BBQ. But that is unlikely to ever happen!

    • Harry Soo says:

      Make a judge cook with a team? What an excellent idea Paul! So before the judge writes down 6 6 6 (deaths) as the score, they think twice, and think again.

      Then they know that’s is a 5 day journey to cook a Saturday contest. One day to shop, one day to prep, one day for travel and setup, one Sat to cook, another travel day home. And that’s a local contest within 6 hours drive.

      For a Kansas City contest, I prep and pack Mon-Tues, leave my driveway in Calif 6 am Wed, arrive in the mid-West late Thursday, cook Fri and Sat, spend Sun to Mon driving arriving home late Monday. And go to work Tuesday, exhausted!

  6. chris hansen says:

    Great article Donna – As a new judge (4 comps) I think the really stupid thing about all of this is that while KCBS will give out the scoring sheets to all of the teams so they can see what their scores were from each of the judges, they wont give the same info to the judges so we can see how we rated the meats we ate compared to our fellow judges. The table talk after each group of entries is not really the same.
    If they really wanted this to be "double blind" they would remove the table and seat number info from the sheets they give the teams and just give them the six scores of their meats and the other 5 teams that were judged by that table.
    Why should they get the info that says it was the judge in seat #4 at table 11 and then look to see what other scores that judge gave and what use is that info? All it does is lead to the kind of flame wars and resentment towards the judges.
    I remember a team at last years Smokin in Mesquite comp that didn’t get a walk with his ribs but won the Peoples Choice with them. As he walked up to the podium he screamed out "TAKE THAT YOU FU***** KCBS JUDGES"

    • Donna Fong says:

      I agree with you Chris. KCBS should consider giving judges access to the contest scores so we can ALL learn. Thanks for the nice comments.

  7. Erick Hacking says:

    This is a great article. As a Master Judge for KCBS with over 60 competitions judged, I can confirm that sometimes you get good meat, and sometimes you don’t, but the joy comes in being there. I try hard to give the benefit of the doubt to the teams, knowing how much goes into that little white box. Sometimes you win, and sometimes you don’t. I love to compare notes with other judges. That’s how we learn.

    There are a lot of great comments here. Lots to think about. But I think the best thing to do when judging is trust your gut (pun intended). If it really is the best BBQ you have ever seen, smelled, tasted or touched, then don’t be afraid to give it a 9. If it isn’t worth more than the original bite, give it a 6. If it has something wrong (like a chemical taste, or too much salt, or dry as a bone, then score accordingly and write a comment. We (the judges) want the cooks to know how to improve just as much as the cooks want to improve and shoot for that elusive perfect score.

    Thanks to all the cooks out there for working so hard, and to all the judges who sacrifice their tastebuds for an afternoon to taste some pretty amazing Q.

    • Donna Fong says:

      I agree Erick. Never be afraid to give a 9-9-9! A 6 to a box means to me that the entry is inherently flawed in some way. But writing comment cards has been very helpful to me personally. They have changed the way I continue to cook. Thank you judges for taking the time to write them! They can mean alot!

  8. Roadrunner says:

    Chris Hansen. When you go to a BBQ restaurant and ask people that are leaving " How’s the BBQ?" a lot of times they will tell you that the ribs are so good, the meat falls off the bone. Peoples Choice is primarily judged by non-CBJs so how can a cook compare that judging with CBJs that judge by a different and set criteria?

  9. Martin Wilde says:

    Great Article – as a former National Beer Judge – I really like the judges comments and strived to provide honest feedback to the beer I was judging. Although I have only done two KCBS Pro Competitions, I am still trying to figure out what the judges want. I was lucky enough to taste an entry that a fellow cook made and I was surprised that it had to set you on fire to get a great score. I did get a call on mine and felt good.

    Please do not take this as a complaint on judging as it it a tough job as I know – just some quick feedback would be greatly appreciated. I know as a judge of other items that taste bud fatigue does factor in

    • Donna Fong says:

      Martin,

      Wow, a beer judge? That’s a good judge to be too! Harry’s article on judging is an interesting trip into the mind of a methodical man if you ask me.

      Donna

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>