The way I see it is that there are two parts of your responsibility in reviewing the required 4 games. The first is to weed out three games so that the organizers have a manageable number of games to sift through in order to choose a winner. This is the aspect of game chef that is a competition and I keep this element separate from everything else. This responsibility is actually really easy. Read 4 games, choose the strongest. If a game is confusing or uninteresting, you don’t really need to keep reading for this duty since it probably won’t get chosen to progress further.
The other part is to critique the games to give feedback to their authors. I have read some such critiques that read like a book report, summarizing the game, and others that use specific systems to judge games reliably, but to me this misses what is most effective in the critique proccess, what will improve the game. I try to focus my critiques on what I think needs to change and what is strong and should remain in grow. I worry less about how much praise I give out, what is good is good, but what isn’t working needs change. Likewise, it shouldn’t matter how playable a game is at this point since the goal is to improve the game no matter what stage of development it is in.
These two principles can interact in very odd ways. The game I recommended to continue was the one that I felt needed the most changes. It also happened to be the game that obviously had the most work put into it, but that didn’t really affect my decision. The reason I chose this game was because it had a concept that I felt best embodied the theme, had the strongest core concept, and was the one that I most wanted to see finished. I had a lot of critiques about the game and huge concerns over it’s implementation, but at the same time I felt this game had the farthest it could go.
So I guess my thoughts are that maybe we need better principles for what needs to go into the separate aspects of our roles.