The  OopslaProgramCommitteeMeeting(s) I've sat on had complex rating schemes for papers :
* your recommendation
* your confidence in your recommendation
* your familiarity with the paper's topic
All of this went into creating a numerical rating for the paper. I found this approach too pseudo-scientific for my taste.

The whole program committee meeting comes down to thumbs up/thumbs down for every paper. I wanted the committee members to be thinking about that binary decision from the first, so I only accepted binary decisions as recommendations. These recommendations went into creating the SortedPaperList. 
-- KentBeck