Simply the Best: Crowdsourcing Choosing the Best Group Amongst Us

Peer evaluation/selection wishes to have a group select a set of “best” members of itself, when everyone wishes to be selected themselves, and despite not having an external objective evaluation. Such a setting happens in refereed academic conferences, when reviewers are also submitting their own papers; it happens in large online courses (MOOCs), when participants are asked to evaluate the work of the other students, etc. Such systems present obvious issues: can we make them strategyproof, so people won’t try to misreport their view of others to improve their own chances or participation? Can we incentivize participation? How can we handle people with bad reviewing capabilities? We show algorithms solving some of these problems, though plenty still remains open.

Based on work with Haris Aziz, David Kurokawa, Harper Lyon, Nicholas Mattei, Jamie Morgenstern, Ariel D. Procaccia, Jeffrey S. Rosenschein, Paolo Turrini, Toby Walsh and Stanislav Zhydkov.

A few relevant papers:
https://tzin.bgu.ac.il/~omerlev/papers/aij23.pdf
https://tzin.bgu.ac.il/~omerlev/papers/aij19.pdf
https://tzin.bgu.ac.il/~omerlev/papers/ijcai15a.pdf