The ENnies are an RPG awards that have become the Official Gen Con Awards. It started out as something small on EN World message boards. It has grown tremendously and become in my opinion the most important RPG awards there are. I do admit to being biased as I was a Judge for the ENnies for six years. What follows are some stories and gossip on what has happened with the ENnies behind the scenes. Some of it is good and some of it is bad. No names are going to be mentioned because while I am writing this to inform people it is not my intention to embarrass or call out any persons. All opinions right and wrong are mine and mine alone.
I’m really not sure why I ran for ENnies judge. The first year I did it I did not get voted in, but I did the second year I tried. I had no idea what kind of hell I was getting myself into. It was fun and I’m very happy I did, but that first year was difficult because of the circumstances. The other judges that year were great. It was a collection of some very intelligent and experienced gamers. We had some great conversations about what should and should not get nominated. The purpose of the judges is to read all the books that publishers submit and come to a consensus list of five books for each of the categories. Once the judges pick the nominations it is put up to a fan vote. In those days it was also the Judges responsibility to create categories that fit the products submitted. For instance we had the best Monster Supplement that year because we saw many monster books submitted. The reason it was hell was almost everything came at the last minute. The last week of May and all of June I had about one hundred and fifty RPGs to read and it is a lot more work than it sounds.
Everything that was submitted was read cover to cover at least once. I had a notebook I would take notes in about each book. It wasn’t just the content we were judging but the production values, interior art, cover art, rules, cartography, and everything about a book. I learned a lot about layout that year. The concept of excessive white space for instance was not something I ever noticed before. I was paying attention to fonts, font size, how pictures were used to help the text flow, or in the case of bad lay out, just interrupted the text. After reading through everything I had a pile of books that I felt were not good enough for any category so I put those to the side. All the others got read for a second and sometimes third time as I reduced my lists of what was best. During this time the judges are communicating online with what they liked and did not like. There were more than a few times that I had to go get a book out of my “no like” pile and reread it because another judge or two sang its praises. Doing this was in addition to everything else one normal does, like go to work. I had a weekly game and I usually was social with friends during the week. To get it all read and to do my job properly I canceled gaming for June. I didn’t see anyone socially and even took vacation time to get everything finished that I had to finish. My girlfriend at the time broke up with me as for that month reading gaming books became her adversary.
Later years became better as companies would not all wait to the last minute to submit books to us. We still felt that too many of them did but that was really to their disadvantage. The books we got after Gen Con and early in the year we had months of discussion on them and more time to read and reread. Books that came in at the very end of the submission period could not get that kind of scrutiny because of the time crunch.
I bring up all the time devoted to reading because we were occasionally accused of not reading everything. One year we were even called out by at the time possibly the biggest name in gaming. Those years I really feel we did read it all. I wasn’t the only one that took vacation time and missed social and family events to get the job done. After we put in all that hard work and had people claim otherwise, that upset us. Some of us blew our tops and angrily replied. I had friends that asked me in private if I really did read everything or if I faked it. I would tell them the truth that I read everything and was insulted that they would imply otherwise. When I signed up to be judge it was a commitment and a responsibility that I had to follow through to the best of my ability. I took it very seriously even though awards are not always that well respected by the fans and by publishers.
In later years when I was and was not a judge I got the feeling that some judges were not reading everything. I’d talk about specific books with them and they were not able to have a discussion about it. Some of the most difficult books to judge are supplements submitted to games the judges don’t own and don’t play. I talked to judges that would read the supplement but not the game it was based on. It’s impossible to judge a supplement if you don’t know the rules and what the supplement is building off of. There is no oversight for the judges. Each judge does their reading in their own place in different part of the US and even the world. They really need to require more transparency for the judges. They should be required to go to Gen Con. At the very least the people running the ENnies should meet with the Judges. Otherwise it is too easy to lie and cheat one’s way into being a judge.
As RPGs grew on the net, other sites devoted to gaming emerged. People had blogs and then podcasts about gaming. I don’t know whose ideas it was to include them in the ENnies as categories but I thought it was a mistake then and still do. As judges we were told we had control of the categories but when some of us talked about not including Best Blog and Best Podcast the Powers that Be told us that was not a possibility. We had freedom when I was first a judge and it worked. When I stopped being a judge much of that freedom had been taken away. My guess it was done so to appease people who disliked the ENnies. We had some publishers that would never enter and then there were game sites that would just talk bad about the ENnies because it is the internet and that’s what people do on the internet.
Podcasts were the worst. It was amateur radio about gaming done in a way that made me want to quit gaming. We had some that were in excess of three hours. People were coughing into microphones. There was dead air. None of the ones I listened to sounded like they were edited or that there was much of a plan going into them. Some were of people actually playing the games. They gave no lead in like what had happened before, what the plot was, and in many cases what the game was. It would literally be thirty minutes before I could figure out what they were playing. I nominated many podcasts based on length of episodes, the shorter the better. Some podcasts tried to be news shows. I would be listening to a podcast in May that were recorded and aired in November. The breaking news was mostly unimportant by then and the guesswork they did on what companies would be doing would be proved wrong. Having to include and listen to podcasts was one of the main reasons I stopped running for ENnies Judge.
Some publishers would enter others would not. Steve Jackson Games I think entered once or twice by accident. At one time I had a contact within the company who I talked with about the awards. He loved the idea and he was ready to submit stuff. Then one day I just stopped hearing from him. When I finally did hear back it was short message that basically said someone higher in the company had told him SJGs would never submit anything to the ENnies and he was told to cease all communication with me. Goodman Games is another odd one. They used to submit modules and some sourcebooks and we would nominate them because they were good books. I heard Goodman Games stopped submitting product because they would rarely win the awards. It’s a fan vote so we have no control over that and we know it is hard to win going up against companies like Wizards of the Coast, Paizo, White Wolf, Green Ronin, and other companies that were just hugely popular. Upsets did happen from time to time. The way the awards are structured being nominated is an award in itself. That is being picked by the experts, the judges that have read everything. If we see seventy adventures in a year and pick one of yours to be top five that’s pretty impressive. Some companies just did not believe it or did not want to see it. Others usually small companies would thank us profusely. They would see a sales bump the week after we made our nominations.
In 2006 we were blindsided by a controversy I was not expecting. I’m still not sure if it was a legitimate complaint or just people airing sour grapes because they lost. Shackled City hardcover was entered by Paizo Publishing. It is a great product that reprints the adventures of the Shackled City AP that appeared in Dungeon Magazine. They expanded on those adventures to include more information especially about the city of Cauldron the setting for most of the Adventure Path. We, the judges, nominated for Best Adventure and for Best Campaign Setting/ Setting Supplement. It won gold for both categories and then some people complained that the book cannot be both an adventure and a setting book. I disagree as there have been books that are both setting and adventures. RPG books can’t always be so easily classified as just one thing.
This past year has also not been without possible problems. When the judges wanted to discuss what should be nominated they wanted to use Skype. It would have been a great way to communicate and I wish we had that ability when I was a judge. But one of the judges refused, and for some reason, instead of majority rules, which is the way it is set up, Skype was not used. Instead the judges had to have the discussions in a less efficient and much slower method of e-mail. That’s a situation that management should be watching over and step in.
Another more alarming example from this year is the exclusion of Star Wars Edge of the Empire Beginner Box. It was submitted but left off all the nominations. It was not left off because most of the judges don’t like Star Wars or felt that other products were just better. It was left off because one judge didn’t like the special dice the game uses and refused to let it get nominated for anything. It is fine that a judge didn’t like the product but no one judge should be able to influence the nominations and have their opinion override the others.
Being a judge for the ENnies was great. I came across a lot of products that I never would have otherwise. I loved that I was there to help open up the awards from just d20 to all RPGs. I think the awards have lost focus on that as we see more items that are not RPGs in them. Voting for the awards is taking place now and is going to be open till July 31st. I encourage all gamers to go and vote. It doesn’t matter if you’ve only read or played a few of the items on there. The more votes that the awards get and the more feedback they get from fans can only improve the awards.
Chris Gath. I’ve been gaming since 1980 playing all kinds of games since then. In the past year I’ve run Pathfinder, Dungeon Crawl Classic, Paranoia, and Mini d6. My current campaign is mini d6 and we are using that for a modern supernatural conspiracy investigative game. On some forums I’m known as Crothian and I’ve written a few hundred reviews though I took a sabbatical from reviewing for a few years as it burnt me out. I was also an judge for the Gen Con awards (ENnies) six times. Jeff, the owner of this blog, is one of my players and a good friend.
My thoughts/comments, as a two-time judge and ENnies volunteer in the off years.
I somewhat disagree about having to be very familiar with a game system in order to judge supplements or, to a lesser extent, adventures. In some ways, it can hurt your objectivity. If it’s a system you really like, your opinion of supplements will tend to skew high or low; you’re either love or hate it depending on how it fits into how you see the game. A little distance is a good thing. Being familiar with the system really only comes into play after a product has made the first cut; a supplement that’s poorly written, organized, and presented isn’t going to be in contention no matter how well it serves the system to which it relates. I always made an effort to at least read the underlying system for those products with which I was unfamiliar (borrowing or buying them on my own when necessary) if I thought they were serious contenders for my support.
I totally agree with you on podcasts, although the latter blur the line a little more. I don’t have a commute or other downtime where listening to people talk about games was a credible solution to boredom. Time listening to podcasts was time taken away from reading, or playing games, or doing other things I enjoyed. Whereas reading and fiddling with new games is something I do regardless of being a judge, evaluating the podcasts was work, and not terribly rewarding work at that. I can barely tolerate listening to rehashes of games I’ve played with people I like.
I also think, for the most part, podcasts, blogs, etc., that are about products rather than actual usable content about gaming don’t belong in the ENnies. A podcast about playing the new Star Wars game isn’t content, it’s advertising or editorializing. A podcast about how to run a horror game is something that’s actually usable.
Companies that don’t submit because they don’t think they’ll win earn my scorn. It’s sour grapes of the worst sort, and it’s supremely disrespectful of the people that pour their heart and soul into a product. A nomination for an art category might not mean jack shit to your company’s bottom line, but I bet it would mean a hell of a lot to the actual artist.
After reading comments regarding podcasts similar to the above (here and elsewhere), I realized I didn’t know what the ENnie awards were specifically awarding. So I would up at the ENnies’ website to find out the mission statement. From their website:
[The ENnies] celebrate the achievements of all tabletop RPGs and the publishers and products that support them.
In that case, “a podcast about playing the new Star Wars game” is a product that supports a specific tabletop RPG. Therefore, that type of podcast should belong in the awards.
As a judge this year, I am shocked at the allegation that Skype would have been better. We must recall that two of our judges are in Europe, and finding a time for all five of us to be online at the same time proved nigh impossible. The method we used relied on Google Drive excel that I personally made, and we were able to tweak the system to its best. It allowed us to work around the clock depending on our global time zone, and allowed us to further discuss games than if we were condensed to a small block of time on skype.
But I did read everything. And reread most. Its a grand experience, and I’m glad to have been able to be a part of it.
With respect, I’ve managed multi-hour G+ calls with people simultaneously in New Zealand, Central Europe, and two US time zones. It’s certainly doable, and as a multi-year judge I strongly believe that it’s essential for bolstering the written process. I’m really glad that the system you set up worked so well, but I hope next year’s judges choose to schedule such a call during the process.
– Kevin Kulp (Piratecat)
Heck, I even spent my own money and vacation time to go visit another judge so we could play some of the stuff that was on the bubble.
Some of us did find several ways to communicate. I personally traveled between tons of OGREs chapters to get different playtest opinions from different types of gamers.
I think that perhaps a system that utilizes what we did with our excel file, probably slightly earlier on, followed by a online chat to go over any not-quite-consensus picks, may work even smoother. It just proved mostly unnecessary this year.
As far as SW – every year people find something to nitpick with excluded items or categories. I’m sure everyone remembers the Free game category fiasco. Each judges panel will end differently.
I must say that this Post — and these comments — have been highly enlightening, I had no idea. Regrettably, I am one of those that has paid little attention to these awards over the years, but this year . . . I voted.
I couldn’t be an ENnies Judge, you guys spend time reading Gaming Materials and Systems that are of absolutely no interest to me. I simply wouldn’t do it. My interest lies solely in Sword & Sorcery type material and games that are based in such a world.
My hat is off to you folks for the work and effort that you put forth on behalf of the community. I can promise you that I will be more appreciative in the future and will be paying closer attention.
Thanks for sharing this with us and letting us “in” on the discussion.
Frankly, Hooper, the fact that you all didn’t convene a live panel to decide on the nominations is disgraceful. You all knew when you signed up that this was a predominately American affair, with all that entailed. I do live chats with people in Europe and New Zealand, there’s no reason given a year to prepare that you couldn’t have scheduled something to accommodate a 5 hour time difference.
The first time I served as a judge we did deliberations with all of us online via Google chat; it took about three hours to has everything out, and there was a ton of excellent debate and discussion. Last year we did it via Google Hangout, and it was even better.
If you can’t be bothered to deal with one late night to actually do the nominations, what credence should we give to the work put into the rest of the process?
5 hour? Wouldn’t that be the /best/ possible time difference?
Seeing as my wife and I deal, daily, with an 8 hour America-Europe time difference and occasionally 10 and it’s very easy to get a 12 hour difference …
Maybe their situation isn’t the one you’ve magicked up in your head?
Several all-nighters, traveled between (I think If I remember them all at the top of my head) 14 different OGREs chapters, and additional online research for products. I know I put in my time. You’re a judge, and from everything I’ve heard a great one, and you had a solid way of doing it. Obviously it worked. I’ll state that I am in no way against a skype or live chat, it just isn’t the linchpin (at least this year) that people pretend. There are several ways to skin a cat, insert other redneck analogies, etc. etc.
“Pretend” is a weasel-word. It’s not a matter of pretending, it’s a matter of firmly believing. It’s fine if you don’t agree with me, Hooper, but please don’t insult the folks whose opinions differ from your own.
I’m not conceited enough to believe that Skype or phone calls are the only way to solve the problem, but it looks like they’ve been more important in my experience than in yours.
Hooper –
Annnd me being rushed came off as me sounding hostile. Not my intent, and while I disagree with you on that one aspect, I apologize if I sounded overly irked.
– K
Pretend is not a weasel word. The ENnies ran for years before Skype existed and they managed just fine and while it is a resource that should probably be used declaring that it MUST be used or you are doing it wrong is just ignorant.
@Micah “Pretend is not a weasel word. The ENnies ran for years before Skype existed and they managed just fine and while it is a resource that should probably be used declaring that it MUST be used or you are doing it wrong is just ignorant.”
Well, 3 years. Then Skype would have been available.
Many people believe actual, realtime conversation is beneficial. And many times it is. I tend towards email on a fairly regular basis, but I certainly see the value in realtime conversation when it comes to more complex matters. And even in cases where I feel like email is appropriate, if people I am working with prefer face-to-face or voice then I wold do what I could to help with that.
Hooper, what about the more damning allegation that Star Wars EotE Beginner Box was dismissed by the grumblings of one judge?
I find it hard to believe that a product that has been receiving a lot of great press on ENWorld and other sites was not good enough to be nominated in ANY category.
That accusation is a lie and the judge that spread that lie to author of this article should be banned from being a judge ever again.
I am confused how you can definitively say the accusation is a lie?
And if there is even a sliver of truth to the original post, is banning a possible whistleblower the way to lead to a more fair and reliable awards process? Or are people really content for the entire process to be this little black box?
Since when is lying about what occurred because you didn’t get your way become “whistleblowing”? I don’t see how spreading false information defaming current judges can possibly lead to a more fair system.
This has nothing to do with getting my way. Hell, if you read any of the Star Wars threads on EN World you’d see I’m not a fan of the game. It is posts like this that sound like liars as you can’t even get basic facts straight.
Crothian, are you the judge that gave this information to the author of this article? Because another judge has already claimed what this article claimed never happened in this very comment section. So did the author just get it wrong? Or did he accurately pass on what was told to him and there is there some other reason besides not getting your way that you are bad-mouthing other judges with false accusations?
Leaving out Star Wars was a mistake, IMO, but there’s almost always one big omission (at least by someone’s standards). “Leverage” not getting a nomination a couple years back was criminal, for example.
That said, as much as I like the new Star Wars game, I can see it not getting noms for game/rules, since it was a crippled intro version of the full product (no character creation rules, for example). But it still should have gotten a nomination for Production Values and Product of the Year.
Again, we devised a system that allowed constant debate and allowed a much more thorough debate on the games. Perhaps a combination of both may be peak, but for the majority of us this year it worked.
In the interest of respect to the system I won’t speak directly on games that were judged this year, outside vehemently refuting the idea that any judge campaigned against SW because of that. A throw away comment discussing it, yes. Butother of us judges lodged different complaints.
I disagree with the idea that the Star Wars Beginner Game shouldn’t be nominated for best rules simply by virtue of being an introductory game. If the rules included serve the purpose of the game/product as best as possible, that is absolutely something that should be considered a *positive* towards a nomination.
The category isn’t “Best Rules in a 300+ page rulebook designed for years of campaign play.”
(This is not to say that I think that it should or should not be nominated this year, because I haven’t played it or the other nominated games.)
It’s further ridiculous given that in 2012 Paizo won 3 awards last year, including Product of the Year, with its intro box (which is a simplified version of the rules)!
It won for Cover Art (which almost anything is eligible for), Production Values, and PotY (which anything that was nominated in any category was eligible for). It wasn’t nominated for Rules or Game, either.
Oh, don’t get me wrong, I *like* the new Star Wars game (enough so that I’m going to be running it in lieu of D&D on Wed. nights at the game store). And I don’t think merely being an intro set should disqualify it for contention in Rules or Game. But I can definitely see it having a harder time competing in those categories against ‘full’ games.
Length isn’t an issue; witness the nominations for Hollowpoint for Rules and Game last year.
Having run the intro set, the lack of Force rules, character creation, and the ‘no dying’ rule were big turn-offs to me, and the spaceship rules seemed incomplete and broken. I can see myself, were I a judge, counting those sufficiently against it to have it not make the top 5. I can also see it going the other way because of the multi-axis resolution system, the party-based action points, and the dice pool mechanics (which I like and loved in WH3) but a lot of people don’t.
Hard to see how it doesn’t make Production Values, though.
That’s what I am saying, Jody. There are so many categories that it could fit into and with the popularity of the game, setting, and looking at the other nominees, I believe it was mishandled.
It doesn’t have to be product of the year, or best rules. What about art, cover, interior, cartography, writing, or one of the other categories?
Art, maybe. Cartography is good, but scant. Writing is always one of the hardest categories to nail down; lots of great stuff doesn’t make the final list in that category.
I would have fought for it for PotY — I think it was one of the better things I’ve seen, and was arguably a better entre to RPGs than the Pathfinder set, which was excellent. But I’ve been there, and know there’s always something good that gets left out when you have to reach a consensus (I’m still upset The One Ring didn’t get more recognition last year…)
I tend to get more upset when something unworthy (IMO) gets nominated.
I once wrote RPG books for a living, with Palladium (and once with WEG), and as a business journalist, I have judged for both the Jesse H. Neal and Azbee awards, which are prestigious awards contests for business publishing. I also have judged the Real LIFE Stories competition, which is published by O magazine. I found the procedures outlined in this article to be pretty interesting. I can feel for ENnie judges since they seem to have a huge volume to go through. Not knowing much about the actual process outside of this article, I would suggest the following, to help things:
1) Have the submissions submit to categories, rather than have the judges determine category. Then assign judges to a specific category and focus just on that. A group for books, a group for podcasts…you get the idea.
2) Establish a submissions deadline and then establish a judging deadline that ensures a minimum amount of time for the judges to do their thing. One month? Two?
3) Use a weighted system where you rank critical aspects of a product (readbaility, editing, art, design, content, etc.) say, 1-10. You can create an online portal for all judging that will compute a raw score from that. Submit your judging.
4) After the first judging deadline, require the judges to have a conference call where they all verbally discuss the top picks with each other to determine the finalists who are then submitted to the fan vote.
5) It goes without saying that as the ENnies assign judges to sections for judging, they have no conflicts of interest themselves (i.e., working for companies that are submitting in that category). Have judges recuse themselves from judging work they are somehow connected to; the weighted scoring system will be robust enough, with multiple judges, to come up with a good result.
I’m a huge fan, Bill. I love your books for Palladium Fantasy. In the 90’s I went by Remmington on the Palladium Boards and was in one of your play by e-mail games.
In the beginning we did have companies submit product into certain categories. A problem emerged where we would read books and see that the book deserved to be considered for other categories. One year Mongoose submitted Slaine for best cover art only and we felt it deserved to be in more. It was nominated for Best cover were it did not win and after I fought for it to get nominated in best d20 game it won the silver. Talking to publishers some felt it was tough to know what categories products could be submitted into.
There is a dead line for product to be submitted and there is a couple of months from then to when the nominations get announced. The time period grew a little bit each year I think so it was a little easier on the Judges.
To avoid any perception of bias Judges are not able to have had anything published in the RPG industry that could be submitted. I’m not sure if that policy has changed or altered but that is the way we always did it before.
The publishers send in copies of the books to be judged so having different judges for different categories would mean Publishers would potentially have to send in more copies. When I was a judge we had a few publishers that were very unhappy to have to send in any copies of their book and requiring them to send in more would not be easy.
Thanks for your comment Bill!
Hey, man! Thanks for your reply. I really need to clarify something. I don’t really have any serious idea of what it means to judge the ENnies. If it’s anything like the journalism I’ve judged, it’s a lot of work, you get second-guessed a lot, and there’s a lot of logistical challenge to overcome. I offered my own experience mainly from the point of reading this article and throwing my own out there. I hope I didn’t come off as trying to throw the ENnies under the bus or anything like that.
I tell you what, though…what are the qualifications to become an ENnie judge? That’s a gig I’d be interested in (though I imagine there are judges aplenty already lined up), thankless though it may be.
It is too late to sign up this year as the voting for the Judges has already started. No current publishers, writers, artists, or anyone actively involved with an RPG company or product can do it. Not as many people sign up as you might imagine. Once they get a list of names it is put up to popular vote which is probably not the best way to do it.
Thanks for the info! I was afraid because I had not published recently, I would not be eligible, but it looks like the inverse is true. I’ll have to keep tabs on this to sign up for next year.
The years I did it, we maintained an online spreadsheet where everyone ranked their top 5 selections. I’m not a fan of trying to nail them down mathematically; for one thing, it tends to penalize those products submitted early (ask any gymnast whether they like to go first in a competition or last..). I kept three (virtual) piles during the year for each category — A,B, and C. Stuff that immediately wow’d me went in the A pile, stuff that was immediately forgettable or memorable in a bad way went in the C stack. Stuff that was in the middle went in the B.
After the submission period was over, I went back and did a cursory re-evaluation of the A and C piles to see if my option had radically changed, then spent a lot more time sifting through the Bs to see if anything moved up or down.
When it came time to meet with the judges for final deliberations (discussions had, of course, been ongoing all year), if something was in my A pile I’d dig in my heels a bit if it didn’t immediately garner enough support, and if it was in the C pile I’d push against it. Stuff that was still in the B pile would garner more discussion.
In general, there was an early majority or consensus for 2-3 products in any category; the stuff that made everyone’s equivalent of the A stack. 90% of the discussions would be about the remaining spots as each tried to convince the others that a certain product was better than the rest.
Really interesting stuff – thanks for posting.
No fixes, just oh the system is broke…six years as a judge, kept coming back. It is a thankless job.
I’m curious what past judges think about the Best Adventure category. How important are the visuals of the adventure? Is it a small advantage, big advantage, or no advantage for an adventure to have great art and layout? How critical is editing? Will a product that has some typos and sub-par sentence structure be automatically tossed aside, even if it would be a fun adventure? How much is the adventure judged purely on the experience it is likely to provide (“players and DM will have an awesome time”) versus these professional factors?
Adventures were always the hardest for me to evaluate. I did pretty good with running games that were in contention so I could see how they played, but it’s just not feasible to do that with 30 adventures in a year (especially if you get a dozen of them the last month…)
Art *always* helps in non-art categories; if there’s a really gripping cover, you’re going to be more interested in reading the product. If the interior art is good and cohesive and thematically appropriate, it reinforces the fluff text.
What I tried to do was get a sense of pacing — does it seem like a lot of pointless encounters, and is the climax a good pay off for the work to get there? Are the individual encounters diverse and interesting — terrain, environment, unusual monsters.
Also important is the layout; is it done in such a way that it will be easy to run, is there good GM advice for ‘what if’ scenarios, maps, handouts, and so forth.
Nitty-gritty details aren’t as important, not because they don’t matter, but with the volume of material to go through, a stat-block error or what-not is probably not going to be noticed. Same with typos or the odd bad sentence. But inconsistencies that affect the ability to run it will — changing the name of the big bad in some places and not others, for example, is a big strike. As I recall, there was one adventure a couple years back that on the surface looked really good and at first blush was on everyone’s short list until someone pointed out a major, game-breaking mistake in the plot text that made it unplayable as written, even though individual elements were pretty good.
This is really interesting, thanks! It does further suggest that organized play adventures can’t really compete within the Best Adventure category under these criteria. Organized Play living campaign adventures are all written by volunteers and thus have to be on a fairly simple and common platform (Word). There is not budget and little time, so there will almost never be any art or professional layout. The editing can be decent, but is more often imperfect. The adventure content can be terrific. Adventures are written by talented individuals, are often very innovative, and undergo rigorous playtesting.
It seems that if Organized Play is to be recognized it would need its own category.
Another former ENnies judge here —
I personally can’t fathom not using Skype, Google Chat or some other at least voice broadcasting tool to go through nominations. There are too many nuances of meaning that you can’t tell through the written word alone.
I’m not saying that what this year’s judges did was *wrong* — only that it would not have worked for me. And even if time zones were a problem, perhaps a meeting with most of the judges is better than no meetings and only email communications.
As for the EotE non-nomination, coming to a consensus is part of the nomination process. Not having a well-reviewed product make the cut is one of the risk factors of having a panel of judges determine nominations. I would venture to guess that many people would critique the 2012 judges (myself included) for not having more nominations for The One Ring.
As for the podcasts, which yes can be quite lengthy — perhaps instead of submitting a specific episode, the rule can be modified “if your submitted episode is longer than 90 minutes, please direct us to the 60 minutes you would like the judges to listen to.”
I wonder if maybe a way to address this in the future is to concentrate judges by category and by continent. The downside there is perhaps North American judges might be into different criteria than, say, European judges. But the time zone difference is indeed significant. I have had to conduct business by phone with folks from the UK, and I’m on the East Coast – that’s a five-hour difference, the shortest you can get between Europe and the U.S., and it still poses significant difficulties. But having said that, I also recall the time when my wife traveled to China, which was 12 hours apart from home, and we still managed to make contact once a day, so it’s possible. It’s just inconvenient as hell. One thing that maybe might be considered is video blogging your thoughts about entries on YouTube, and just restrict access to invites only. That way, even though the conversation won’t be live, you can still communicate a more nuanced discussion of a particular entry. It would not be a perfect solution, but it might possibly offer more context than a purely written exchange. Just a thought.
That really wouldn’t work for most things; Art, Writing, Rules, Adventure, Supplement, Cartography, Game, Production Values all basically cover the same products. Podcasts and blog are really the only two that don’t have much overlap with real products, and if you’re going to have separate judges just for that, may as well just make them their own award.
All of that said, as someone who submits, reviews and judges academic works as part of my duties as a faculty member, I find that the review process is very difficult on both sides of the equation. Sometimes you submit something and maybe it gets a reviewer who is not fond of the work or doesn’t see the merits of what you have done, while another individual would have loved it. Judging is not something which is perfect, but I believe that if you do good work and are persistent you will get your rewards, though it might not be in the form of a prize, but more in general recognition from the community. I think EotE has that from a lot of people, so maybe prizes are not the most important thing.
Going back to the reviewing/judging side of things. It is a difficult thing. Sometimes I’m asked what kind of contribution a work makes to my field, and I’m told to give a score of 1,2,3,4 or 5. But really, what does this score even mean? In the end, it comes down to the comments I write, as well as the collective final recommendation of several referees. The goal in this process is to try and eliminate as much judging bias as possible. No judging system is perfect, but people do their best.
With the ENnies, something that doesn’t happen is that the judges don’t share why the nominations were selected. I wonder if there would be value in naming the strengths that the product had. For me, it would be valuable to learn from the judges. Was adventure X chosen because of innovative concepts? Because it provided a deep imaginative story? I think more could be learned by the industry and gamers if the strengths were listed.
Similarly, it would be helpful to know the criteria by which products will be judged. What goes into deciding what makes a great free product? What goes into evaluating a great adventure? Are the do’s and don’ts? Are some aspects more important than others? (For example, does the quality of art in a free product matter at all, or is it expected that this category would be low-art and low-layout? Is a free product that is basically advertising, such as a free quick-play pdf from a major publisher judged differently than one that is stand-alone, such as a free simple RPG created by someone for fun?)
Companies do have a basic understanding of criteria, and they can request categories though they are never limited to just that category if they do happen to qualify for another.
Though there are several valid reasons that presenting why each one made the cut would be great, there is also a big case of risk management. What if the judges pool presents privately to each company a very nice list of what they liked and didn’t, then a third party gets a hold of said list and completely sensationalizes it to a incorrect “ENnies judge say X sucks, don’t buy it” type of thing. Hopefully it wouldn’t happen, but look at this blog here – all good people involved with it, but there is still a hugely sensationalized claim (that one judge snowballed SW because of dice when in reality they simply listed it as a personal Con they had for the game, and that particular judge then never discussed the game again). Risk management is important.
As someone who has submitted something, I have no idea what the criteria may be. Where can I find that information?
With regards to sharing strengths, I meant that this would be public. Example: “We really liked how Gardmore Abbey introduced dynamic NPCs, which could be allies or foes based on the actions of the PCs. We also appreciated the production value of the components, including maps, tokens, and the Deck of Many Things.”
While people complain about anything, the benefits to both industry and consumers from sharing strengths would be pretty huge. I look at the list of nominations and I have no idea what those products are about, what their strengths are, why they were nominated, etc. I don’t come out of this smarter as a consumer or as a freelancer. It is only if it creates discussion, entirely separately from the ENnies, that I might learn why a product is worth checking out.
That’s a question that comes up among the ENnies folks every year (and I’m sure this year was no different), but I don’t think the upside outweighs the downsides. I think it would just encourage more nerd rage and nitpicking of the ‘How can you say module X was a good adventure when there was a stat block error on page 27?’ variety. It’s judging; it’s inherently subjective.
I think asking judges about what criteria they would use (and I’ve answered questions along those lines) is fair and a good idea during the time the judges are running for election, though.
Isn’t the yearly (and almost always poorly-attended) judge’s panel at Gen Con where questions about “why did this product get nominated” could get answered?
I think that we (2012 judges) were the first attempt at a panel, and given the lack of attendance, likely the last.
But there was almost a full handful of people at that panel! Of course, they were all friends/associates of the ENnies…
It is interesting to me that the OP, and several subsequent respondents, don’t seem to feel as if there is any room, or need, to recognize the amateur enthusiast.
Given the statements above, I’m reasonably certain ENWorld itself would be criticized as being too amateurish to win an award, if such a circumstance were possible.
Thank god the hobby was built by serious minded professionals and not just some folks who loved gaming and worked out of their garage.
The awards do a great job of recognizing Amateurs that get into publishing with nominations in certain categories. They have two categories this year of Free Game and Free Product which aside from last year’s debacle is usually a great place to see them. With the prohibitive cost of publishing the best electronic Book is also a great place for Amateurs that get into the business.
However, the Free Product includes offerings from very large companies (Privateer, Paizo, Catalyst). That’s a category where I would be curious to know the criteria. Will an amateur be able to compete with the art and layout of the bigger companies or their design, development, and editorial staff?
There is no need to recognize the amateur enthusiast simply because they are an amateur or enthusiastic. Put out a great product, and it will get noticed. But it’s damn hard to put out a great product if you’re the writer/designer/artist/editor and all the playtesters were your friends. If your game is more than a few pages, a professional editor and completely unaffiliated playtesters are essential.
If you’re suggesting that there should be different criteria for ‘quality’ based on the resources of the person making the game, I think that’s nonsense. The ENnies aren’t golf; you don’t get bonus points for being a weekend duffer, and they don’t hand out participation ribbons just for showing up.
Actually, in particular, I’m referring to the general dismissal of pod casts and blogs as being anything like worthy of ENnies recognition.
You do yourselves a disservice, and really show yourselves as short-sighted, when you write them all off. A vast majority of game content (unofficial or not), advice, and discussion comes out of them. Far more than any official publisher has ever produced for any system.
Sure, some of it is of questionable value and quality, but to simply state that all of them are unworthy of your time and consideration smacks ever so slightly of elitism and makes it seem as if some judges feel they are the guardians of all that is right and proper in gaming.
The job of a judge, it seems to me, is to find the best of what IS submitted, not to automatically refuse entry or consideration based on personal preference, inconvenience of material, or some high handed idea of how games should be used, played, or enjoyed.
You are either finding the best of what you have, or filtering what the public has access to so that the options match your personal preference.
Podcasts are not a good way to give game content. That should be done on a page be it in print or a web page. It is hard to go back to a podcast and find a bit of information. I think the ENnies just need to focus on RPG books be they physical or electronic and products that are designed to use with gaming. There are a lot of different Axillary products that move the awards away from their focus and can really bog down the judges.
Podcasts should have their own awards. They have grown and there are so many of them that they could organize and have awards for different categories of Podcasts.
They did that in the Academy Awards with what is now called “visual effects.” They gave a separate recognition to the visual effects/engineering of movies. Now they are considered an important part of the movie industry.
In the same manner, I think that blogs and podcasts should be given equally important consideration. Maybe casts that are just recordings of five people playing a game with no intro or review is nothing special, but that doesn’t mean all the good content should be excluded. I think the judging should demand excellence from podcasts and blogging, so that these media forms will continue to improve. In that way, they will be important parts of the game industry which garner the recognition they deserve.
I disagree strenuously with Chris and Jody; I think podcasts are (or can be, at least) great. I’ll suggest that just because you don’t care for something doesn’t negate its value as a gaming aid.
I definitely hope to keep seeing podcasts in the ENnie Awards. I think podcasts can fill a niche and be useful to people that just like to listen to people talk about gaming, or even interviews with people in the industry.
I would think podcasts could be handled somewhat efficiently. In the examples of poor editing, that should be pretty evident fairly quickly. I’m not a judge, but if folks are coughing into mics, lots of echo, etc, that is probably going into the ‘C’ pile pretty quickly without listening to a full podcast.
I also don’t quite get the Actual Play podcasts either. Frankly, I would probably compare all of those to The Delvers. If an actual play wasn’t in the ballpark of that one, it would probably slide to the ‘C’ pile as well.
I think that’s fine, but I’m hearing two things (amateur’s are recognized / they can’t compete). It isn’t clear. The criteria aren’t known, so it isn’t understood whether an amateur can compete in the free categories. Amateurs won’t usually have great layout, art, and development/editing, which is why they often offer things for free. But, we also see major companies bring all of their considerable talents to bear in creating free products as advertising. What are the criteria for these categories?
The Criteria is if it is a free Game it qualifies otherwise if it is a free supplement to an RPG it would qualify for that. One thing the ENnies does is treat the small press guys the same as the big boys. The ENNies doesn’t care if a book was produced by a 100 person team or a one person team. When I was a judge I talked to a lot of small press companies and they said they liked it that way. If we have categories that the big companies can’t compete in then it’s like saying this is the best of rest. The ENnies wants to showcase the best of the best. It does make it tough but occasionally the small press guys win and that is a very meaningful win for them.
This was a fascinating glimpse into this process. I have suspected other awards of being heavily slanted, it never dawned on me that the ENnies were. I guess it shouldn’t come as a surprise. I believe this article will go a long way to starting to change the process.
I wanted to acknowledge that I am very thankful for all the hard work ENnies judges do. While I do have questions and do see ways the ENnies could better serve gamers and the industry, I recognize how hard a job this is to do. Thanks!
This part is a blatant lie:
“Another more alarming example from this year is the exclusion of Star Wars Edge of the Empire Beginner Box. It was submitted but left off all the nominations. It was not left off because most of the judges don’t like Star Wars or felt that other products were just better. It was left off because one judge didn’t like the special dice the game uses and refused to let it get nominated for anything. It is fine that a judge didn’t like the product but no one judge should be able to influence the nominations and have their opinion override the others.”
The judge you mention said he would not nominate it but if everyone else liked it he would not argue. This is documented in the emails involved. You are either misinformed or blatantly lying.
This seems a little more like “he said, she said” to me. Your evidence is?
The emails exist. There is a record of what was said.
I do not know who Micah is but I assume that I can correctly guess which judge he is a friend of and that it is the judge who made the statement about the dice. The conversation really went something like this: “Judge A lists pros and cons of the game, judge b does the same, judge c” etc. Judge A just happened to list the dice thing as a con. At no point did that judge strong arm or even attempt to persuade others against their opinions. Hell, when this blog came out 4 out of 5 of us had to go back through the messages just to know what the hell it was talking about.
Furthermore, skype was never actively prevented. It was brought up, we went with the google drive excel system first due to the extremely different time zones. Then we did the talking and debates and before we knew it we had a list and never had to do a final skype or conference call.
It sounds like the blogger here just jumped the gun after talking with one of this years judges about going ons and didn’t have a clear picture of everything, or took one judges opinion as fact for a pool of people. I just hope that in future years that doesn’t happen. Its not “whistle blowing,” its just assuming without confirming.
Thanks for your level-headed replies throughout, Hooper. I appreciate your participation in the comments here.
Three hour podcasts? Coughing in the mics? Unprofessional tripe like that shouldn’t be on the internet, let alone win an Ennie.
I totally agree, listening to something I’m interested in for more than 15 mins upsets the ‘ol ADHD. The fact that these rank amateurs don’t take the time from their everyday to lives to edit these ‘social’ forums to a professional standard is deplorable. I truly believe that forum posters should, at the very least, be able to communicate with the written medium at an undergraduate level as such I would expect no less from presenters of the auditory medium.
If I have to listen to one more foolish podcast about off topic subjects like favorite beers, what someone had for their dinner, games that do not involve the d20, or why F.A.T.A.L. should not win an ENnie I think I will start to unravel. There should be a limited number of ENnie sanctioned RPG podcasts that are limited to under an hour and have proven that they can remain on topic. All of these various ‘amateur radio’ ramblings are destroying this hobby.
Who are these self appointed guardians of the hobby to dismiss the THE MOST REALISTIC RPG EVER PRODUCED. I’m talking about F.A.T.A.L. It’s disgusting that they refuse to review it. My character sheet is fully nine ages long with real information I can use not silly feats like d20.
Gosh – those are some strong words against podcasts.
I couldn’t disagree more. Any enthusiasm from the fan-base can only HELP this hobby.
Especially when they showcase products/systems/what-have-you and spread the word about what their hosts find value in. In fact, that creates demand — something that benefits everyone, from companies creating products to people looking for others to game with.
As for your opinions, it’s rather easy to avoid podcasts if you do not want to listen to them. So don’t listen — seems easy enough to me. And leave those who do enjoy podcasts to their entertainment.
But he’s a former Judge, like Judge Dredd he is The Law and he is there to decide what we should like for us. Like so many disillusioned Judges of Megacity 1 he has taken The Long Walk into the Cursed Earth to bring Law to the Lawless. That is to say he wishes to bring his enlightenment to we the ignorant
Pingback: Haste! Red Aegis, ENnies Expose, Shadowrun and more! | Words In The Dark
F.A.T.A.L. really does need its own special ENnie…as for podcasts, beer drinking should not happen on them either, even though podcasts, especially the long ones (It’s like speeches – some short ones are good, some long ones are good, yet they should not be judged on length but the content) that are much fun. I am saddened reading this what these awards do represent and will stand aside from them.
Pingback: Ennies 2013: A Rogue Warden’s Critique | My CMS