Best Paper Committee Chair Handbook
Current Chairs:
| Name | Affiliation | Contact |
|---|---|---|
| Rada Mihalcea | University of Michigan | mihalcea@umich.edu |
| Roi Reichart | Technion - Israel Institute of Technology | roireichart@gmail.com |
Overview of Responsibilities
The Best Paper Committee Chairs oversee the selection of conference awards (e.g., Best Paper, Outstanding Paper, Area Chair’s Award) in compliance with the ACL Conference Awards Policy (see here).
The following are the main responsibilities of the Best Paper Committee Chairs:
1. Forming the Best Paper Committee
2. Managing Nominations and Reviews
3. Organizing the Selection Process
Breakdown of Responsibilities
1. Forming the Committee
This responsibility focuses on recruiting appropriate committee members for the determination of "Best Paper" and "Outstanding Paper" awards.
| Task | Start | Deadline | Dependency | Notes |
|---|---|---|---|---|
| Contact Previous Year’s Chairs | ASAP | April 2025 | - | Gather insights on committee composition and process. |
| Create List of Potential Committee Members | ASAP | April 2025 | - | Prioritize diversity in research areas and demographics. |
| Invite Committee Members | March 2025 | April 2025 | - | Aim for a large enough committee to maintain a load of 10-15 papers per member. (2) Committee should be formed before the submission deadline. |
2. Managing Nominations and Reviews
Reviewers*, Area Chairs (ACs), and Senior Area Chairs (SACs) must be asked the following questions pertaining to the nomination of papers for awards: 1. "Could the camera-ready version of this paper merit consideration for an "outstanding paper" award (up to 2.5% of accepted papers will be recognized in this way)? Outstanding papers should be either fascinating, controversial, surprising, impressive, or potentially field-changing. Awards will be decided based on the camera-ready version of the paper." [Yes/No/Maybe]\ 2. "If yes/maybe, please briefly describe why:" [Short answer box without word/character limit]
* Publication Chairs may decide whether to include reviewers in nomination decisions.
| Task | Start | Deadline | Dependency | Notes |
|---|---|---|---|---|
| Collect Candidate Papers | Post-Senior Area Chair recommendations | May 2025 | - | (1) Include papers marked "Yes" or "Maybe" by reviewers/AEs/ACs/SACs. (2) Program Chairs may decide whether or not to include papers marked "maybe" by a review but not nominated by the AE/AC. (3) SACs may also nominate a paper in their area for the Area Chair Award (this must not be given to a paper that one of the SACs have a conflict of interest on). (4) The Best Paper Committee should not be told which papers have received an Area Chair Award, to avoid biasing decisions. |
| Confirm Eligibility | Before notification | May 2025 | - | (1) Check for pre-publicized papers (e.g., arXiv). (2) Paper are ineligible for awards if their authors include Program Chairs (and PCs may also choose to specify that authors in other senior organizational roles are also ineligible for awards). |
| Distribute Anonymized PDFs | Camera-ready deadline | May 2025 | - | Redact author info if needed (PDF). |
3. Organizing the Selection Process
The process below was developed based on the assumption that the nomination process leads to 50-80 papers being considered for awards. If the number of nominated papers turns out to be much higher, this process will not scale. Best paper award chairs have latitude and discretion for the design of the selection process. PCs should approve the process.
Whilst a specific rubric is not defined, committee members should consider the following aspects when reviewing award nominees: 1. Is each paper either fascinating, controversial, surprising, impressive, or potentially field-changing? Note that papers do not need to demonstrate all of these properties; any property is sufficient. 2. Does the paper present as its motivating use case an application with significant negative social impact? Even if the motivating use case is an application with neutral or positive social impact, are there obvious applications with significant negative social impact which are left unaddressed or insufficiently addressed by the paper? (In such cases, the paper should not be an award candidate.) 3. Is the work presented in the paper reproducible? For example, is there sufficient information in the paper to repeat the experiments? If not, is the lack of reproducibility justified in the paper? 4. Do the awards highlight a broad range of research types and strengths? 5. Do the awards include types of research that can be conducted at small labs? 6. Do the awards include papers that show excellence in potential positive social impact?
- Long/short papers should be considered jointly for awards.
- Additional "Best Paper" and "Outstanding Paper" award slots are available for non-publicized papers.
- Please find additional information pertaining to award criteria here.
| Task | Start | Deadline | Dependency | Notes |
|---|---|---|---|---|
| Allocate Papers | May 2025 | May 2025 | (1) Committee members should receive anonymized camera-ready papers, anonymized reviews, and associated supplementary material. (2) Best Paper Committee chairs are encouraged to redact author information on the first page of papers. (3) The committee should be discouraged from searching for revealing author information from different sources. (4) Each paper read by ≥2 members, based on research areas. | |
| First-pass Reviews | May 2025 | May 2025 | - | Papers should be categorized as: (1) consider for best paper, (2) consider for outstanding paper, (3) no further consideration. |
| Final Deliberations | May 2025 | June 2025 | - | (1) All committee members must read papers that both committee reviewers placed under consideration for "best paper". (2) No more than 0.25% of accepted papers must receive a "Best Paper Award" (inc. main and Findings) - the limit is 6 papers for conferences with under 2000 accepted papers. (3) No more than 1.5-2.5% of accepted papers must receive an "Outstanding Paper Award" (inc. main and Findings). |
| Announcement, Certificates and Financial Awards | June 2025 | Pre-Conference | - | (1) In recent editions, all awardees have been announced directly during the awards session on the third day of the conference, in order to maintain suspense. Only awardees scheduled to give a short presentation will be notified in advance, and in any case, before the conference begins. (2) All awardees will receive a certificate with the name of the conference, name of the award, title of the paper, and author names, signed by the Program Chair(s) and/or General Chair(s). (3) Conferences may choose to have a financial component for any of the awards, which may be sponsored, and may be split between the authors as the authors choose. |