Search the Community
Showing results for tags 'Judging'.
Found 3 results
This wall of text post is the result of a private conversation I have had with someone here on the forums. I have edited it a bit and removed names and things of that nature. Hopefully it may answer some of those questions people can be afraid to ask. I'm happy to have conversations like this and I thought that this might spur some more questions that people are uncomfortable asking, It starts off like this Let me ask you honestly, do you think that some things are judged more harshly for the genre they are painted in such as some may like say historical better than fantasy? Also, do you feel there are ever biases for people who've gotten awards in the past or are "well known" or "friends of" the judges? Reason I ask is because I saw some and looked very closely at a few figures and things that I got "dinged" for seemed to be overlooked on figures which got higher marks. I'm not calling anyone out or trying to start trouble, but I was a bit put off by how varying the standards were and I talked to a few friends who had certain judges that'd mark at say a Bronze and one would mark at a Gold citing that the bronze didn't like the technique, but the gold uses it and loves it. I felt some were inconsistent. Really, I just want to know how to "level up" and be able to achieve higher standards with how varying the judging is. It's a bit confusing at this point. Those are hard questions for early in the morning, quite frankly they are hard questions period because we are dealing with people in a subjective environment. I'll do my best to explain it. As you know a team consists of three judges who after deciding which piece to score (if there are multiple entries in a category) then assign a score of 0-4. Michael and I do instruct the judges (and we are usually on teams as well) spending about 15 minutes refreshing everyone on what we are trying to accomplish. In general judges on a team don't consult with each other on what score they are going to give, they will consult with each other if they are struggling on how to score something. In general Michael and I both tell them to start with silver score (a 3, we also try to avoid giving 0s at all, except in the youth division) and then move up or down from that point. Do judges have a bias against different genres? While I don't believe that judges have particular biases between genres there are certainly areas where judges are stronger or weaker. For instance, we have judges that we won't put on a team that will cover vehicles and ordnance because its weak area for them. However, they are excellent when judging in the painters category. I think it has more to do with how comfortable a judge is within a category rather than a bias towards a particular genre. Do judges have a bias for past award winners, or friends? This is a really tough question and its one of the reasons there are three judges on a team and not a single person doing it. There is a kind of a yes and no answer to this question. As judges we do expect a higher quality of work from a past gold medalists, Sophie winners, best of show winners or even instructors. In general they probably get judged a bit more harshly because of their past work, we know what they are capable of. Do we maybe allow some of them to dial it in anyway, quite possibly, again it depends on the judging team and quite frankly when someone like Jen Haley or Jess Rich dials it in they are still head and shoulders above the rest of us. Friends are a bit of different story, in general we tell judges that they need to bow out of judging a piece that they have a vested interest in because it was done by a friend or because they consulted a significant amount on it (I was pulled into this myself this year when they were judging the Dark Sword awards). We always have alternate judges available to fill in when something like this happens and it is not an uncommon occurrence. Unfortunately, it is not always clear when something like this has happened because you have to see the physical judging sheet to see if a different set of initials are in the scoring box for that piece. [Edit: Judges don't score their own pieces either, this is another place where the alternate will step in.] Are judges inconsistent? [this was my initial answer "To a certain extent yes". As I reread this I have changed my mind]. Judges tend to be very consistent with how they score things (I have been tracking judges' scores for years now), what is inconsistent is that no two judges view entries in quite the same light, we are human after all. We make mistakes or we are grumpy because we have been on our feet all day teaching and that can be unconsciously reflected in how an entry gets scored. While Michael and I try to catch the obvious ones like a judge scoring a 4 and another judge on the team scoring the same entry a 1 scores like a 1, 2, 3 (which adds up to a bronze) are not uncommon. We are dealing with someone's opinion on what they are looking at. Remember, in general, the judges don't discuss how they are going to score a piece with each other. If there are multiple entries the discussion is more along the lines of "I can score this piece higher than that one". We try very hard to encourage discussions in positive terms not negative terms. That doesn't mean it doesn't happen and many of our judges have a background in trophy judging where the first thing you are looking for are flaws. We want our judges to look at the good things first. I can't speak to the last question because I would have to see the circumstance. Most judges don't put down comments on the scoring sheets (we can, but often time gets away from us). For your particular example I would have to know what entry is being talked about and what category it is in and who the three judges were. Also what was the third score? I do track the judges' scores and we try and make sure teams the teams are balanced. Some judges in general are tougher than others. I don't want two "tough' judges on the same team nor do I want to two "easy" judges on the same team either. In general, I think that the results are for more consistent than most people realize but unless you are right there doing the scoring and involved with the conversations it is hard to see that. And yes, we have had individuals that are not particularly good at judging despite how good a painter and instructor they may be. For that one year, they are going to have an influence on the entries they score in the competition. The judges, like the entrants, are human and have their own inherent biases and opinions. By using three judges we hope to achieve a balance and get realistic scores for every piece that was entered. With our need to get more judges into the pool we did run a bit of an experiment this year. In vehicles and ordnance, I judged the category with two newbies to the ReaperCon competition. However, they are both experienced judges in that category at other shows. I was pleased to find that our scores were almost in lock step with each other with a few 1 point variations which I would expect to find. Will they do as well as we introduce them into the painter, open, and diorama categories? I donâ€™t know, but I would be comfortable using them as dedicated vehicle and ordnance judges now. This lets me use a third judge that is not familiar with the category get experience under their tutelage. I also have people asking how they can become judges. A hard question and not one that you asked. We like our judges to have some experience as instructors first. The first thing we need from a judge is the ability to communicate with people and be familiar explaining techniques and able to give good critiques. The easiest way for us to do that is make sure they are good instructors first. It doesn't always mean that they are going to be good judges but it gives us a starting point. The first year a new judge serve as an alternate. They are typically assigned to a single team for that year and step in when a primary judge needs to step out. That way they get the advantage of being involved in the discussions and can get in and score a few things themselves. The year after that we try and work them in as a primary judge on a team, that is usually a make or break year for us to figure out if they are going to be a good judge or not. I'm not sure that I have answered your questions very well. It is hard to sit down, based on the judging, and figure out what you need to do to "level up". We emphasis the feedback portion of the competition but sometimes that can cause confusion when you are getting hit with multiple opinions. I think the best critiques start out with the judging asking you what you think you did really well and what you didn't like. Often you know what needs to be done you just haven't acknowledged it. Thanks for taking the time to write out such a great response, Kris! I really do appreciate it. I know it's very subjective, and all judges are human (well some may machines with how well they paint!). Really I guess it comes down to conflicting feedback from judge to judge for me. Some may love your blending and use of colors and give high marks, others may think it's crap and give you low marks. I was just curious if there are "standards" they should follow to try to keep personal feelings on techniques used out of the equation. A good example of this is drybrushing. I know a few judges who quite literally sneer at the usage and feel it's something that shouldn't ever be used and others who respect it as a tool in the toolbox to be strategically used if done well. There is not a set of standards. It would literally be impossible to set something like that unless we selected a single artist to be our standard, so we have to lean on experience and skill. To a certain extent you are always going get different opinions from the judges and that's were their own bias towards something is going to get in. It doesn't mean that they necessarily dinged you for that for scoring, unless they were quite specific about it. We do instruct the judges that all techniques are valid, that they are judging the execution of that technique even if it is a technique that they don't use or don't think is effective. If you executed drybrushing correctly then you shouldn't get dinged for it even if the judge doesn't like that technique. Oh, no problem, it's mostly curiosity on my part at this point after talking to a few other friends and hearing their scores/feedbacks. Again, this wasn't about my particular judges for painters category. I got 2 bronze and 1 silver scoring so it was pretty consistent. I also got feedback from others like Jen Haley which was very valuable. And I'm not trying to cause trouble or stress, really want to be able to understand the system better so I can do better and know what people are looking for as they judge pieces as someday I would like to get golds and Sophies. I know that my biggest critiques are about contrast and I get it, but I personally like more subtle things like the Europeans are doing. Things with textures and grittiness. But I also know I should conform a bit because right now in painting in the US it seems to be all about the contrasts and smooth blends for judges. Again, thanks for your time in talking about this and I really do appreciate the thought you put into your responses. Many of our judges have been exposed to the European styles and have taken classes with the European painters. They should be able to appreciate that style as well. Just keep in mind that in some ways the Europeans take contrast, especially between light and dark even more extremely than we do. Really there are a heck of a lot of styles out there now. It just feels to me like the biggest thing people look for in these contests is contrast and always upping values to the extremes. I get it, these are tiny pieces and the lighting isn't the best to judge by so subtly will easily get missed in competition when you are looking over hundreds of entries and you're tired and have certain things each judge looks for. I always welcome conversations on how to get better with painting and composition and techniques so a full forum conversation could be pretty good for everyone. I was a bit disappointed in myself for only getting bronze in painters as I felt that I leveled up on my pieces since last year when I also got bronze. I got a bunch of feedback from a lot of people on the forum and other places on my techniques. Really, I do just want to get better and am open to all advice and comments in order to do so. Taking classes and getting personal feedback are great ways to do this and part of the reason I love ReaperCon so much. The leveling up in medals is really hard. One of the things that you have to remember is that while the scoring sounds simple: 0-1 - no award 2-4 - Certificate 5-7 - Bronze 8-10 - Silver 11-12 - Gold the difference between all those scores is much bigger than it appears. The difference between scoring a 7 (high bronze) and an 8 (low silver) from the judging point of view is significant. I didn't do a lot of critiques this year since I only judged one category, but the single biggest thing I saw this time around was thick paint. Keep it thin! While many of us are restricted to asking for advice and help online, it is hard to really to really get and give good feedback. Things I see in a picture may not be the result of your paintwork but more the result of your lighting. What looks great in person may look awful in a photograph and vice versa. Ultimately there is no magic bullet, you need to paint, paint and paint some more to get better. Which means that most people aren't painting enough to get from one level to another and if you take long breaks between painting you forget things and skills deteriorate. Its practice, practice, practice! Also if you are consistently painting for the table top you are in the habit of painting till its good enough. For a competition piece that is not good enough, you need to practice to a competition level to get better if you want to level up from bronze to silver or silver to gold. I remember reading about a Canadian painter (and he has been at ReaperCon in the past, but his name escapes me) who went to his first GW Games Day in Canada and decided the next year he would win the slayer sword. He succeeded in doing that, he also spent almost 400+ hours researching, painting, striping and re-painting that piece. He may have started off that year as a newbie painter but by the end of it he was master. Most of us don't have that kind of time, but it does show what can be done if you are up to the challenge and are willing to put in the work. There is no magic bullet to getting better except to keep painting and improving.
So you are coming to ReaperCon and its coming fast, especially for those of us still working away on our entries. ReaperCon is a rather unique convention and there is nothing else quite like it out there. Since the focus is on miniatures and painting this is a good convention to enter into the painting competition especially if its your first time. Now that sounds scary I know, you have heard that some of the "big" names are going to be entering why should you bother? ReaperCon's painting competition is in a much friendlier format than most (not all, but most) game convention painting competitions. Its a good place to get your feet wet. You can check out the rules on the ReaperCon.com website. But really what does all that mean? First let's take a look at the categories, there are only four of them. Why four? All the other shows seem to have a dozen categories. We have modeled this competition from the one used by MMSI, which is also used by a good chunk of the military/historical painting shows. The idea is that you don't need a dozen or so categories when we aren't going to award a 1st, 2nd and 3rd place (sometimes referred to as podium or trophy judging). Instead we want to reward you for the hard work you have put into your entry. Instead of 1st-3rd we award gold, silver, and bronze medals along with certificates of merit, this is known as the Open (or medal) system. Since we are going to reward you for your work we don't need a dozen categories to give people as many chances as possible to win an award. Instead we want you to focus your painting on what you do best and four categories is all we need. The titles can be a bit confusing so let's take a look at them. Painter - The focus here is on a single stock miniature. While presentation (i.e. basing) is a factor an elaborate base isn't what the judges are focusing on. I use the term stock here because essentially you are using the mini right out the package. Conversion work whether its elaborate or simple is not considered, although a poorly done conversion can hurt your score. Open - The focus of this category at ReaperCon is a bit different than it is at an historical show so be aware of that if you go to MMSI in Chicago or Lone Star here in Texas. What is acceptable for ReaperCon may not qualify as Open there. Our Open category is primarily on the conversion of existing miniatures, scratch sculpts and elaborate basing. Painting is still a factor but its not weighed quite as heavily as it is in Painter. So if you have spent as much time on the base as the miniature and want it considered as part of your score then this is the category for you. Dioramas/Vignettes - While this category is pretty standard at game convention competitions, at MMSI and similar shows these typically go into the Open category. This category is focused on story telling through the use of miniatures and basing. There are different ways to define dioramas and vignettes. The simplest I have seen is that a diorama has more than 3 figures on the base and a vignette has 3 or fewer figures. Painting is still a component of this category but the emphasis is on presentation in the sense that you are trying to convey a story or message to the viewer. Keep it as simple as you can, while I have seen some wonderful complicated dioramas out there sometimes there is so much action they muddy the story. Keep this mind, if you have to explain your story to some one looking at your scene then you have failed to convey your message. A diorama or vignette needs to stand on its own and convey the story without explanation from its creator. Here is a clue, if you can't figure out a title for your diorama, then you may not know what your story really is. Vehicles/Ordnance - This category is for those things of a mechanical nature, subject to a bit of interpretation. A horse drawn wagon is a vehicle. In this case the wagon and its team of horses would be judged as they are the "vehicle", while the riders are not considered for the painting portion but would be considered as part of the presentation. It can get a little complex. So what about this medal stuff and how do I know I won? You can think of the medals as a grade awarded by a team of judges. What the judges don't do is compare your work to the entry right next to you (which may actually be judged by a different team anyway). The judges will score your work without comparing to other entries. If they decide you have earned gold, then you will receive a gold medal for your entry. You can only win one medal in each category. It works like this; each piece is scored by each of the three judges on the team independently of each other (so you get 3 scores). If you have multiple entries in Painter they will discuss which piece they are going to judge. They do not discuss what score they are going to award a piece. After determining the piece to be judged each judge assigns a score from 0 (yes Zero) - 4. When they have finished working all the pieces on their list those sheets are handed in and one of the staff members totals everything up to determine the score. So not even the judges know what what the final score for a piece actually was until the awards ceremony. Those three scores for your piece are added up to determine what medal you receive: 0-1 no award, 2-4 Certificate of Merit, 5-7 Bronze, 8-10 Silver, 11-12 Gold. The Award Ceremony is Saturday Night. Hey! The rules say I can enter as many miniatures as I want, why can I only get one medal per category? Yes, you can have as many entries as you want in each category. As mentioned above though the judges will only score one of those entries in that category. An initial conversation is held to determine which piece will be scored, its often along the lines of "I can score this miniature higher than that one". Judges will score the piece that they think is your best work (which may not be what you consider your best work, it happens). So while you can certainly enter all "20" single miniatures you painted this year in the competition you are only going to get a medal for one of them. The judges will go through this process for each category, hence why the maximum number of medals you can receive is 4. There are other awards as well, the Sophie Trophy, the Theme Award, and various manufacturer awards. These are judged separately and use the more traditional 1st - 3rd method (in essence that can be boiled down to this miniature has fewer painting flaws than that miniature). Its possible that a single entry could win multiple awards. The judges do have the prerogative to score your entire display if they can't reach a decision on a single piece or they feel that the display of miniatures, as a whole, is worthy of being rewarded with a medal. What was my score and why did I get it? After the awards ceremony and when the painting competition hall is open you can ask not only what scores your mini received but who judged it. Most of the judges are taken from the ranks of the instructors at ReaperCon with a couple of exceptions (myself for one, although I do teach on occasion) so your miniatures are being judged by people that are knowledgeable about painting and how to do it. We use teams because we feel (and its one of the reasons MMSI developed this system) that a combined score is more indicative of what a mini should get rather than depending on a single judge to know everything and be neutral on all the different painting techniques that are out there. To find out what the judges were thinking you will have to track them down. Most of us are more than happy to discuss the whys and wherefores just be aware of our time and that you may be taking up the only 15 minutes we have to eat. Be considerate. Some Hints for Entering 1) Every piece has to have a name or title. To speed up registration please already know the name or title before you get to the front of the line! Write it down before hand if you need to! As I mentioned before if you don't already know the name of your diorama or vignette you may have an issue with your story. 2) Don't bring everything you painted in the last six months. Yes, I know it says unlimited but really if you painted "20" miniatures this year is the first one better than your last three? Odds are the last three or four are probably more indicative of your best work. Try to keep you numbers down to around 5 or fewer per category. 3) Make sure your bases are at least finished in the Painter category. While presentation is not a huge chunk of the percentage in this category a nicely finished base will show off your miniature better than the base you tried out different color combinations on or used to wipe excess paint off your brush on. 4) Make sure your entry is well fastened to its base, you don't want to be subjected to the "Heisler Affect". If you mount your mini on a pedestal style base, judges tend to see that as a handle. If the mini is not attached when its picked up by the "handle" its going to hit the table, probably to disastrous effect. 5) Make sure the paint is dry when you hand in your mini for the competition! 6) You must enter all your miniatures at the same time. You cannot bring them in as you finish them in the painting room. So if you have 5 entries for each category then you have to bring all 20 entries at once, not a couple at a time. 7) Remember that if you have ReaperCon Full Weekend badge your entries must be in by 5pm on FRIDAY night. No exceptions. 8) If you have a Saturday only badge your entries must be in by 12 Noon on SATURDAY. No exceptions. If you have a ReaperCon full weekend pass you cannot enter on Saturday you missed your cutoff. 9) Displaying your work. There are things you can do to help your pieces stand out, especially if you have multiple entries. We do have a limited supply of blocks and black felt so you can separate your minis out from the crowd a bit and do a multi level display. Please be aware of how much room your taking up or a judge may come by and consolidate your display if you are taking up 2' of space for 4 minis. Bringing your own display backdrop (again spacing!) will make your minis standout. While this isn't going to have an affect on your score (judges usually pick the entries up) it will draw people in to look at your work. 10) Blending. I have had a couple of questions about blending. Blending doesn't necessarily have to be perfect it depends on the type of technique or painter you are trying to emulate. For instance Alfonso "Banshee" Giraldes' technique or style doesn't require perfect blends it does require excellent understanding of the play of light and the shadows it creates. So this is likely to be pretty subjective. If I think of other things I'll add them to the list.
This is the second in a series of four posts each concentrating on a different entry category. You can find information about the scoring system itself in the Painter Division post. From here forward I will just concentrate on how the component guidelines apply to the other three divisions. Open Division The Open Division is far more of a freeform division than the Painter Division. Here is where you get to really strut your stuff with major conversions and scratch sculpts. If you have just a single entry then the judges can just go ahead and score your entry, no discussion is necessary. If you have multiple entries, then there will be a discussion between the judges on which entry they want to score. That conversation is typically the only conversation although these discussions tend to be longer than they might be in the Painter Division. However, when selecting the scoring entry the conversation is still based on “I can score this one higher than the others” or words to that affect, till they come to a decision just as it would be for the Painter Division. If multiple entries are visually very thematic the judges may decide to judge them together as a single entry. Let’s take a quick look at the scoring guidelines the judges use (which is published as part of the MSP Open rules): Difficulty: 15% Creativity: 10% Workmanship: 30% Painting Skill: 30% Presentation: 15% What does this mean for the Open Division? In this division we are really want to see all your skills. While the components remain the same the emphasis has obviously changed a great deal. Difficulty: This is far more intuitive than it is in the Painter Division. The level of difficulty depends entirely on the difficulty of the conversion, with a minor conversion being the least difficult with graduations on up from there with a complete scratch sculpt being the most difficult. Creativity: Creativity stays about the same as it does for Painter. Painting is still a factor here. Now painting is combined with your ability to convert and sculpt to reach your audience. The entrant’s imagination comes into play here, you are looking for impact on the audience. Are you straining the boundaries of believability or are you trying to evoke a specific emotion from your viewers? Have you achieved what you set out to do at the end? Workmanship: This remains a pretty straightforward component but in the Open Division there is a higher emphasis on it. It reflects how well constructed the entire piece is. Any type of non-painting effort is represented here. Again a well done conversion means that the judge can’t tell that anything has been converted. A scratch sculpt should be properly proportioned and well sculpted (no thumb prints!). A missed mold line, poor assembly or a poorly executed conversion could easily drop you a while numeric value in the scoring. This is a category that we really encourage documentation, show us what you did and how you did it. Painting Skill: Everything that was said about painting still applies in the Open Division but there is less emphasis. At this point workmanship and painting are equal. While we don’t expect your abilities to be exactly equal in both areas you cannot count on your ability to paint alone to carry you over the top. Presentation: There is more emphasis on the is component as well. If you are building the entire entry, essentially from scratch then the presentation of your entry is going to have a significant impact on how a viewer perceives your entry. Bring your entry to life!