Newsgroups: rec.arts.int-fiction
Path: nntp.gmd.de!Dortmund.Germany.EU.net!Germany.EU.net!EU.net!howland.erols.net!netcom.com!erkyrath
From: erkyrath@netcom.com (Andrew Plotkin)
Subject: Re: Review of Russ Bryan's Review of Andrew Pontious's Capsule Reviews of Contest Games
Message-ID: <erkyrathE1yK66.n0u@netcom.com>
Organization: NETCOM On-line Communication Services (408 261-4700 guest)
X-Newsreader: TIN [version 1.2 PL1]
References: <5826jv$1h8@news.thorn.net> <cleofax-0412960720040001@noho-us252.javanet.com> <kjfair-ya023280000412960050500001@uchinews.uchicago.edu> <5841as$k5h@vixen.cso.uiuc.edu> <Pine.SUN.3.91.961204120437.6292A-100000@nebula.phy.duke.edu> <584il7$6r5@vixen.cso.uiuc.edu>
Date: Thu, 5 Dec 1996 20:42:06 GMT
Lines: 91
Sender: erkyrath@netcom.netcom.com

> What I mean is, to create a common attributes that can be agreed upon 
> for most stories that can be used as a guidelines to ensure that every 
> story gets evaluated at some basic level similarly.  For example, the 
> attributes might include:

> 	plot
> 	writing style
> 	interactivity
> 	confusion
> 	bugs
> 	creativity
> 	...and so on

Nope. I deny this set of criteria, and all sets of criteria. I have a 
criterion, which is "Did I like it?" This is totally vague and subsumes 
everything that is important to me -- but not by computation, because 
that's not the way I rate games.

(I know I said a lot of this early in the voting period, but, well, the
subject has re-arisen. :-)

> that would not be forced on judgements but used as "Hey, make sure to 
> consider all of these things," or judges could rate each game on each by 
> their own choosing.  These criteria I have laid out here are common to 
> all stories with few exceptions and if a story was so original that it 
> did not fit them, well of course the criteria become less useful.

I am most interested in the exceptions, and I strongly disapprove of any 
mandated criteria that does not take them into account. Which means *any* 
fixed set of criteria.

I definitely agree with this reply:

> >Beyond the impossibility of coming up with a set of 
> criteria 
> >acceptable to all (I suspect GKW would be forced to impose one by 
> fiat), 
> >I find it valuable that everyone brings a different viewpoint to the 
> >competition.  Forcing a rigid set of criteria on the competition could 
> >serve to stifle certain types of games.

(Yeah, I've lost all the attributions now, blame TIN.)

> The problem I see is what I notice in the reviews.  With one exception, 
> what is being evaluated changes /per game/ by judge, not to mention 
> across judges.  Given this, ranking all of the entries becomes a joke.  
> Unique perspectives are important, but the authors deserve a common set 
> of basic, minimal criteria.

They *have* a common set of criteria: *What the judges liked*. This year, 
people went to all *sorts* of different places to try to be liked. Some 
worked, some didn't. 

Are you counting me as the exception? It may not seem like it, but I 
*did* have a single criterion, as I said. Not only that, but it's the 
same one I use in the real world. (Modulo some issues about using hints 
and finishing games, which are for another discussion, please. :-) I rate 
games as I would enjoy them if I just ran across them on GMD.DE. I think 
this is a fine basis for a competition. In what other sense are we 
competing?

I think you may be confused because my *reviews* did not comment on the 
same things. But my scores were not based on my reviews. Honest. The 
reviews were more for the benefit of the authors than anyone else, and 
focussed on the interesting (or interestingly failed) aspects of each 
particular game. The scores were separate.

> Right now what is happening is a hodge-podge of scores based on 
> everyone's individual criteria.  While that sounds democratic, it's also 
> a big muddy mess.  While there appears to have been a clear winner, I 
> dismiss any ranked differences between the rest of them as noise.

If you're going to worry about that, you have to worry about the much 
larger skewing effect of normalization -- what counts as a "1" and what 
as a "10"? That has nothing to do with the criteria. I wound up saying 
that my favorite game of the entries that I played was a "10", and my 
least favorite was a "1". But other people might have used a scale where 
"10" was the best game *they'd ever played*, and "1" the worst.

I won't even get into linearity. Those of you who know what a gamma 
factor is (in video displays) can consider its applicability. I tried to 
spread the entries evenly along the 1..10 scale, which meant that "7" was 
not exactly 7/9th of the way from "1" to "10". I said I wouldn't get into 
this. I'll shut up now.

--Z

-- 

"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the
borogoves..."
