I had a hard time finding any write-ups on other people’s real world experiences of preparing for and taking the BJCP Judging Exam ( aka BJCP Tasting Exam ) so I figured I might as well put my experience out there for people preparing to take the exam. Take it all with a grain of salt – I obviously in no way represent the BJCP on these matters and these are just my own thoughts on the BJCP certification process and judging in general. The first half is a lengthy rant on my experience preparing for the exam and the second half is the real meat where I discuss how I prepared for and the results from my exam.
Rant:
First, some background. I started homebrewing about 4 years ago and have been entering beers in competitions for about 3 of those. I’m not totally sure where I first got the bug to get BJCP certified except that a lot of homebrewers I have a lot of respect for recommended it. Initially my goals were to develop my palate so I could more critically assess my own beers and to get the opportunity to formally judge other brewer’s beers hoping I could both learn from it as well as do something positive for the community at large.
A lot of my expectations and initial understanding of how beer judging works were totally wrong and have dramatically changed as I’ve gone through the process of getting certified. When I first got started, I had some interesting ideas on beer judging. As an engineer by trade, I sometimes fool myself into thinking that real world processes can be perfect. That somehow, through super secret underground training regimes certified beer judges have uber palates and can definitively discern the difference between a 31 and 32 point beer. And when I first started entering homebrew competitions, I thought if I entered a beer in many competitions it would get very similar scores each time. It was only after entering many beers, many times with wildly divergent scores (20 this week, 44 next week – really) and judging at competitions with other judges and being 10 points apart and having to figure out a common ground that I realized how incredibly wrong I was.
So, a lot of the initial value I saw in entering beer in competition and judging was blown-out of the water. I started to feel a bit disillusioned and I still feel like the whole process is flawed for one big reason: preference. Every judge has different preferences and no matter how hard you train yourself to judge by style and technical merit, at the end of the day if you don’t like munich malt or citra hops or a hint of DMS in your lager then you just aren’t going to score those beers as high. I don’t think any judging program can fix that since we are human and having preferences is in our nature. And if they weren’t, we would all go to the store and buy the single 50 point beer that put all the others out of business.
That said, I think the BJCP program and it’s current certification program are very well setup and really do drive judges towards two key goals that should be top of mind while preparing for the exam:
1) Scoring within 7 points of other judges
2) Properly filling out a scoresheet
Scoring accurately is self explanatory but getting in the zone is hard. More on that later. Properly filling out a scoresheet also sounds easy but there is a lot involved. Being able to evaluate a beer and properly perceive the level and qualities of the aroma, appearance, flavor and mouthfeel. Properly describing what you’re perceiving using a common language. Having a firm grasp on the style guideline so you can properly ascertain if the beer is to style and if not where it misses the mark. Knowing enough about flaws and brewing procedures to provide accurate feedback on how to correct any perceived issues.
And it’s not perfect. Sometimes I’ll judge and never have to adjust a score and sometimes I feel like I’m having to adjust every score because the other judge and I are so far apart. Heck, on my exam the two proctors were ten points apart on a fresh from the store, commercial example beer. And these were two very experienced national level judges! We are all human. It happens. But with more training and experience, it happens less and less. Also, I’ll often judge with certified or national level judges who will repeatedly tell me they are looking for a certain quality in a beer – more chocolate in a dusseldorf alt, more roasted barley in a northern english brown, more caramel in a DIPA – that are fine at very low levels but certainly shouldn’t stand out or be something you are looking for. Maybe after judging for many years, people forget to look at the style guidelines or we all begin to think we know something cold and forget that everybody needs a refresher now and again.
But after finally taking the exam and getting my scores back – although not as high as I wanted – I realized that the exam really was quite good and did push me directly towards it’s two goals. I only scored half my beers within 7 points of my proctors and so I got dinged for it. And I should have. I didn’t comment on every aspect of each section of the scoresheet and used imprecise language and so I got dinged. And I should have. So, even with a bit of disillusionment with beer judging in general, when I look back and see how much I have gained it seems silly to be in any way upset. I can now pick-up off flavors and aromas in beers that two years ago I’m sure I perceived but had no idea what I was perceiving and how to describe it. I now really get what all eighty beer styles are supposed to be like and can easily tell when a beer just isn’t quite right and why. And I am now very confident in my ability to score properly and fill out a scoresheet that I would be very happy getting back from a competition. And I guess that is really the point. I want to be a part of the judging community so that when someone takes months to brew a beer, ferment it, cellar it, package it and then pay money and somehow get it to the competition that they know it was properly evaluated and that the feedback that they get back is accurate and useful. Because when I enter competitions, that is what I want.
Over time, my reasons for judging have definitely shifted. Before it was mostly for personal gain – learning how to critically assess beers – namely mine. Now it is more for the community. I’m not a perfect Beer Judge, but if I don’t judge then it is more likely that a local bartender who “drinks a lot of damn good beer”, a local brewer who puts an average of 10 words per scoresheet or just some club members friend ends up judging your or my beer. That’s not critical, expert analysis and feedback. That’s just a waste of a lot of time and $6. I do see a lot of value in having competitions where your beer can be properly judged and be compared to your peers and given props when you’ve done a great job and you can get totally independent feedback on any issues and some ways to maybe fix those issues. And if I see value in that and I want there to continue to be good competitions, then it is incumbent on me to do my utmost to be the best judge I can be and to participate.
Meat:
I had a really hard time getting signed up for an exam. I had four exams within a three hour drive but all were full and my only option was the waiting list. So, I got on a few of those and finally got confirmed for an exam about 9 months out. Of course, the week before a bunch of earlier exams I would get the email saying that they had a last minute opening but I stuck with the one I was confirmed for. So, that gave me 9 months of prep time. In those 9 months I made sure I had read every book on the recommended reading list (page 3 of http://www.bjcp.org/docs/BJCP_Study_Guide.pdf), that I had had at least one commercial example of every beer style and I had critically assessed it and that I judged as much as possible which was maybe 3 different competitions.
In retrospect, I think the most important of all was sitting down with examples of each style and really making sure I got the style and what they meant by each of the descriptors. My process was to get as many commercial examples as I could find and then with each one I would read the style guide, then judge each beer the same I would in a competition – scoresheet and all – and then reread the style guide while going through my scoresheet. I also got together with a local group of other people training for the same exam and we would go through a similar process but without scoresheets. I think the scoresheets would have been good even for that scenario in retrospect. The other thing I had been working on since I started brewing was brewing all 80 styles. I’m about halfway through now but that has been an enormous benefit. There are just so many different ingredients and brewing techniques that until you have actually used them it is hard to perceive the impact and describe how to fix any issues or praise a job well done. So, lesson #1:
Critically judge multiple commercial examples for each style
One thing I did wrong in preparing was searching the internet for info 🙂 In all seriousness, I had read some posts from people saying they had been given doctored beers. This really made an impression on me and hung in the back of my mind the whole lead up to and during the actual taking of the exam. And on two beers, it caused me to be way off from the proctors. That leads me to lesson #2:
Judge the beers the same way you would normally
Even if the beers are doctored (which none of mine were – wish I had known that ahead of time!), don’t forget one key fact – the two proctors are judging these blind the same way you are and how they judge the beer is what matters. So just judge as you normally would and don’t go hunting for some doctored defect. For me, I had heard sometimes beers are watered down so even though my doppelbock was a commercial example that is on the low end of the spectrum it was still to style. And in a normal competition I would have scored it in the mid to high 30s rather than the high 20s and been within range. For the sweet stout, I was convinced they had given us a purposely low-sweetness beer when in fact it was just the least sweet of the commercial examples for that style and I had never had it. My impression of a sweet stout was Mackesons which is WAY sweeter than American versions.
Another lesson learned was to keep in mind how your exam is being scored. The basics are for each beer there are 100 points and there are 20 points for each of the following – scoring accuracy, perception, descriptive ability, feedback and completeness. There are two national level or higher judges out there who get a couple proctor score sheets and your score sheets. They don’t have the beer. With this huge limitation they are going to score the following way:
1) how many points are you off from the proctors average – goal is within 7 and the closer the more points you get
2) pick out some key perception aspects of each beer from the proctors score sheets and see if you hit those aspects – for example grainy malt aroma, low diacetyl, high carbonation etc… These would be items that stand-out about the beer.
3) KEY ITEM: make sure you have level and description for each item listed below the section header. For example, on a scoresheet you see Aroma and below that it says “comment on malt, hops, esters and other aromatics”. For each of those they will look to see that you gave a level “medium malt” and a description “grainy, biscuity”. Make sure you cover ALL of them this way.
4) accurate and useful feedback. if you point out flaws or style misses you should give ways to fix the issue and they should be technically accurate and not TOO specific. keep in mind, you don’t know anything about how this beer was made. So saying to use fresher extract would be silly. Or more amarillo hops. How do you know they used extract or amarillo hops. No matter how amazing you think you are at perceiving extract twang or tangerine hop aromatics, you don’t know how the beer was made so don’t comment as if you do.
5) kind of a gimme – everything should be filled out and should be legible. white space = bad.
I will say that even if I was seemingly good on 2-5 but my score was off by more than 7 points that everything else suffered score-wise. So, even though it appears that you can do well with inaccurate scoring (it’s only 20% right!) that is not the case. In the scoring guide it states you need to have 3 correctly scored beers to be in the 70s, 4 to be in the 80s and 5 to be in the 90s So don’t think you can be way off on scoring and still do well. Which brings me to lesson #3:
Read the Scoring Guide (http://www.bjcp.org/docs/BJCP_Scoresheet_Guide.pdf)
Reading the Exam Guide is pretty obvious, but it never crossed my mind to read the Scoring Guide and understanding how you are going to be scored is pretty key 🙂 And by reading the Scoring guide you can see how focused things are on the two goals I laid out above. They are really trying to make sure that the body of BJCP judges score within a reasonable range of each other and can fill out a killer scoresheet that any entrant would be happy paying an entry fee to get.
So now on to my actual exam. The six beers I had were German Pilsner, Weizen, Dusseldorf Alt, Tripel, Doppelbock and Sweet Stout. I felt pretty good since I had brewed four of these styles and was very familiar with commercial examples of each. Kind of sucked for anyone who didn’t like German Style beers!
Here is my overall summary:
Rant:
First, some background. I started homebrewing about 4 years ago and have been entering beers in competitions for about 3 of those. I’m not totally sure where I first got the bug to get BJCP certified except that a lot of homebrewers I have a lot of respect for recommended it. Initially my goals were to develop my palate so I could more critically assess my own beers and to get the opportunity to formally judge other brewer’s beers hoping I could both learn from it as well as do something positive for the community at large.
A lot of my expectations and initial understanding of how beer judging works were totally wrong and have dramatically changed as I’ve gone through the process of getting certified. When I first got started, I had some interesting ideas on beer judging. As an engineer by trade, I sometimes fool myself into thinking that real world processes can be perfect. That somehow, through super secret underground training regimes certified beer judges have uber palates and can definitively discern the difference between a 31 and 32 point beer. And when I first started entering homebrew competitions, I thought if I entered a beer in many competitions it would get very similar scores each time. It was only after entering many beers, many times with wildly divergent scores (20 this week, 44 next week – really) and judging at competitions with other judges and being 10 points apart and having to figure out a common ground that I realized how incredibly wrong I was.
So, a lot of the initial value I saw in entering beer in competition and judging was blown-out of the water. I started to feel a bit disillusioned and I still feel like the whole process is flawed for one big reason: preference. Every judge has different preferences and no matter how hard you train yourself to judge by style and technical merit, at the end of the day if you don’t like munich malt or citra hops or a hint of DMS in your lager then you just aren’t going to score those beers as high. I don’t think any judging program can fix that since we are human and having preferences is in our nature. And if they weren’t, we would all go to the store and buy the single 50 point beer that put all the others out of business.
That said, I think the BJCP program and it’s current certification program are very well setup and really do drive judges towards two key goals that should be top of mind while preparing for the exam:
1) Scoring within 7 points of other judges
2) Properly filling out a scoresheet
Scoring accurately is self explanatory but getting in the zone is hard. More on that later. Properly filling out a scoresheet also sounds easy but there is a lot involved. Being able to evaluate a beer and properly perceive the level and qualities of the aroma, appearance, flavor and mouthfeel. Properly describing what you’re perceiving using a common language. Having a firm grasp on the style guideline so you can properly ascertain if the beer is to style and if not where it misses the mark. Knowing enough about flaws and brewing procedures to provide accurate feedback on how to correct any perceived issues.
And it’s not perfect. Sometimes I’ll judge and never have to adjust a score and sometimes I feel like I’m having to adjust every score because the other judge and I are so far apart. Heck, on my exam the two proctors were ten points apart on a fresh from the store, commercial example beer. And these were two very experienced national level judges! We are all human. It happens. But with more training and experience, it happens less and less. Also, I’ll often judge with certified or national level judges who will repeatedly tell me they are looking for a certain quality in a beer – more chocolate in a dusseldorf alt, more roasted barley in a northern english brown, more caramel in a DIPA – that are fine at very low levels but certainly shouldn’t stand out or be something you are looking for. Maybe after judging for many years, people forget to look at the style guidelines or we all begin to think we know something cold and forget that everybody needs a refresher now and again.
But after finally taking the exam and getting my scores back – although not as high as I wanted – I realized that the exam really was quite good and did push me directly towards it’s two goals. I only scored half my beers within 7 points of my proctors and so I got dinged for it. And I should have. I didn’t comment on every aspect of each section of the scoresheet and used imprecise language and so I got dinged. And I should have. So, even with a bit of disillusionment with beer judging in general, when I look back and see how much I have gained it seems silly to be in any way upset. I can now pick-up off flavors and aromas in beers that two years ago I’m sure I perceived but had no idea what I was perceiving and how to describe it. I now really get what all eighty beer styles are supposed to be like and can easily tell when a beer just isn’t quite right and why. And I am now very confident in my ability to score properly and fill out a scoresheet that I would be very happy getting back from a competition. And I guess that is really the point. I want to be a part of the judging community so that when someone takes months to brew a beer, ferment it, cellar it, package it and then pay money and somehow get it to the competition that they know it was properly evaluated and that the feedback that they get back is accurate and useful. Because when I enter competitions, that is what I want.
Over time, my reasons for judging have definitely shifted. Before it was mostly for personal gain – learning how to critically assess beers – namely mine. Now it is more for the community. I’m not a perfect Beer Judge, but if I don’t judge then it is more likely that a local bartender who “drinks a lot of damn good beer”, a local brewer who puts an average of 10 words per scoresheet or just some club members friend ends up judging your or my beer. That’s not critical, expert analysis and feedback. That’s just a waste of a lot of time and $6. I do see a lot of value in having competitions where your beer can be properly judged and be compared to your peers and given props when you’ve done a great job and you can get totally independent feedback on any issues and some ways to maybe fix those issues. And if I see value in that and I want there to continue to be good competitions, then it is incumbent on me to do my utmost to be the best judge I can be and to participate.
Meat:
I had a really hard time getting signed up for an exam. I had four exams within a three hour drive but all were full and my only option was the waiting list. So, I got on a few of those and finally got confirmed for an exam about 9 months out. Of course, the week before a bunch of earlier exams I would get the email saying that they had a last minute opening but I stuck with the one I was confirmed for. So, that gave me 9 months of prep time. In those 9 months I made sure I had read every book on the recommended reading list (page 3 of http://www.bjcp.org/docs/BJCP_Study_Guide.pdf), that I had had at least one commercial example of every beer style and I had critically assessed it and that I judged as much as possible which was maybe 3 different competitions.
In retrospect, I think the most important of all was sitting down with examples of each style and really making sure I got the style and what they meant by each of the descriptors. My process was to get as many commercial examples as I could find and then with each one I would read the style guide, then judge each beer the same I would in a competition – scoresheet and all – and then reread the style guide while going through my scoresheet. I also got together with a local group of other people training for the same exam and we would go through a similar process but without scoresheets. I think the scoresheets would have been good even for that scenario in retrospect. The other thing I had been working on since I started brewing was brewing all 80 styles. I’m about halfway through now but that has been an enormous benefit. There are just so many different ingredients and brewing techniques that until you have actually used them it is hard to perceive the impact and describe how to fix any issues or praise a job well done. So, lesson #1:
Critically judge multiple commercial examples for each style
One thing I did wrong in preparing was searching the internet for info 🙂 In all seriousness, I had read some posts from people saying they had been given doctored beers. This really made an impression on me and hung in the back of my mind the whole lead up to and during the actual taking of the exam. And on two beers, it caused me to be way off from the proctors. That leads me to lesson #2:
Judge the beers the same way you would normally
Even if the beers are doctored (which none of mine were – wish I had known that ahead of time!), don’t forget one key fact – the two proctors are judging these blind the same way you are and how they judge the beer is what matters. So just judge as you normally would and don’t go hunting for some doctored defect. For me, I had heard sometimes beers are watered down so even though my doppelbock was a commercial example that is on the low end of the spectrum it was still to style. And in a normal competition I would have scored it in the mid to high 30s rather than the high 20s and been within range. For the sweet stout, I was convinced they had given us a purposely low-sweetness beer when in fact it was just the least sweet of the commercial examples for that style and I had never had it. My impression of a sweet stout was Mackesons which is WAY sweeter than American versions.
Another lesson learned was to keep in mind how your exam is being scored. The basics are for each beer there are 100 points and there are 20 points for each of the following – scoring accuracy, perception, descriptive ability, feedback and completeness. There are two national level or higher judges out there who get a couple proctor score sheets and your score sheets. They don’t have the beer. With this huge limitation they are going to score the following way:
1) how many points are you off from the proctors average – goal is within 7 and the closer the more points you get
2) pick out some key perception aspects of each beer from the proctors score sheets and see if you hit those aspects – for example grainy malt aroma, low diacetyl, high carbonation etc… These would be items that stand-out about the beer.
3) KEY ITEM: make sure you have level and description for each item listed below the section header. For example, on a scoresheet you see Aroma and below that it says “comment on malt, hops, esters and other aromatics”. For each of those they will look to see that you gave a level “medium malt” and a description “grainy, biscuity”. Make sure you cover ALL of them this way.
4) accurate and useful feedback. if you point out flaws or style misses you should give ways to fix the issue and they should be technically accurate and not TOO specific. keep in mind, you don’t know anything about how this beer was made. So saying to use fresher extract would be silly. Or more amarillo hops. How do you know they used extract or amarillo hops. No matter how amazing you think you are at perceiving extract twang or tangerine hop aromatics, you don’t know how the beer was made so don’t comment as if you do.
5) kind of a gimme – everything should be filled out and should be legible. white space = bad.
I will say that even if I was seemingly good on 2-5 but my score was off by more than 7 points that everything else suffered score-wise. So, even though it appears that you can do well with inaccurate scoring (it’s only 20% right!) that is not the case. In the scoring guide it states you need to have 3 correctly scored beers to be in the 70s, 4 to be in the 80s and 5 to be in the 90s So don’t think you can be way off on scoring and still do well. Which brings me to lesson #3:
Read the Scoring Guide (http://www.bjcp.org/docs/BJCP_Scoresheet_Guide.pdf)
Reading the Exam Guide is pretty obvious, but it never crossed my mind to read the Scoring Guide and understanding how you are going to be scored is pretty key 🙂 And by reading the Scoring guide you can see how focused things are on the two goals I laid out above. They are really trying to make sure that the body of BJCP judges score within a reasonable range of each other and can fill out a killer scoresheet that any entrant would be happy paying an entry fee to get.
So now on to my actual exam. The six beers I had were German Pilsner, Weizen, Dusseldorf Alt, Tripel, Doppelbock and Sweet Stout. I felt pretty good since I had brewed four of these styles and was very familiar with commercial examples of each. Kind of sucked for anyone who didn’t like German Style beers!
Here is my overall summary:
First up is the German Pilsner:
My scoresheet:
My scoresheet:
RTP (Report to Participant – in case you hadn’t noticed the BJCP TLA use level is FUBAR):
Comments: Seems that I’m either really sensitive to Acetyldehyde or I was just wrong. Then again it says the judges picked up “low apple flavor” so maybe I was right and it was an oversight on the scorer. Learned that specific descriptors are better than vague statements – ½” head or medium head rather than good head. I definitely over used the term good throughout the exam. For some reason my scorers were very into bubble size. Seems silly to me but duly noted.Weizen:
My scoresheet:
My scoresheet:
RTP:
Comments: The main thing I learned from this beer is a beer has to be truly terrible to be in the teens. I can still taste this beer in my mind. It was really, really bad but I guess not bad enough to warrant a 15 cause it must have been scored by the proctors in the mid-20s or higher. It had no head, undercarbonated, crystal clear, ester/phenol profile was off and it has a distinct sour note throughout the flavor. so it pretty much missed all the key style markers and on top of that had a unpleasant off-flavor. I really think this was too generously scored by the proctors but oh well. The main issue with my scoresheet is I didn’t comment on all items which is a consistent theme and definitely something I will be focusing on.Dusseldorf Alt:
My scoresheet:
My scoresheet:
RTP:
Comments: More of the same. I didn’t give level and description for everything listed for each section. Forgot that if I point out a flaw I need to give recommendations on how to fix it. Appears my score was right on.Tripel:
My scoresheet:
My scoresheet:
RTP:
Comments: This beer was just god awful. I’d rather purge it from my memory. Not sure what it is with me and sour notes but maybe I need some calibrating there. Also made the same mistake of pointing out a flaw but not describing how to fix it.Doppelbock:
My scoresheet:
My scoresheet:
RTP:
Comments: This was my first of two commercial example beers that are on the low end of the style in some regards but certainly too style and I was convinced they had been doctored. My gut told me this was a high 30s beer but I was convinced there was supposed to be a style flaw so I then told myself I couldn’t go higher than high twenties. Lesson learned: go with your gut. Other than that, more of the same – I didn’t give level and meaningful descriptors for every “comment” item and fell short on feedback.
Sweet Stout:
My scoresheet:
Sweet Stout:
My scoresheet:
RTP:
Comments: And I think this is where the alcohol started to kick in. Same story as Doppelbock. I was convinced this was not sweet enough for the style. This is mostly because I had only had British commercial examples to that point and they are super sweet. Oh well.
Overall:
Overall:
Comments: By far my biggest takeaways from the exam are:+focus on giving level and meaningful descriptors for every single comment item in each section
+give feedback for every flaw listed
+go with your (trained) gut when scoring. At this point, I know what a 15, 25, 35 and 45 point beer is and shouldn’t get too hung up in trying to find doctored beers or other silliness to find reasons to score in a certain range.
The whole process was very professionally run and even if I don’t totally agree with everything said the scoring was very thorough and overall very well done and, too bad for me, a fair score was given (a hard fought 71 🙂 ). I hope anyone reading this finds it helpful. And yes, you won’t get your score for a long time. Just forget about it and one day it will show up in the mail. If it’s been a year, maybe you should email someone.
+give feedback for every flaw listed
+go with your (trained) gut when scoring. At this point, I know what a 15, 25, 35 and 45 point beer is and shouldn’t get too hung up in trying to find doctored beers or other silliness to find reasons to score in a certain range.
The whole process was very professionally run and even if I don’t totally agree with everything said the scoring was very thorough and overall very well done and, too bad for me, a fair score was given (a hard fought 71 🙂 ). I hope anyone reading this finds it helpful. And yes, you won’t get your score for a long time. Just forget about it and one day it will show up in the mail. If it’s been a year, maybe you should email someone.