Instructions for Reviewers

Thank you for reviewing for EMNLP-IJCNLP 2019! In order to ensure the quality of reviews, we would like to share with you the following instructions for reviewing EMNLP-IJCNLP 2019 papers. Please read these instructions before you start reviewing papers.

Confidentiality

Please note that the content of any submission to EMNLP-IJCNLP 2019, and the participants in and content of discussion on submissions, are confidential.

The Review Form

We adopted the review form from ACL 2019 and made some minor changes. The review form consists of three main sections: 1) In-Depth Review, 2) Questions and Additional Feedback for the Author(s), and 3) Confidential Information.

  1. In-Depth Review: This section is for you to give your overall assessment of the paper and to provide evidence to support your opinions. There are 5 subsections in this section: (1) What is this paper about, what contributions does it make, and what are the main strengths and weaknesses? (2) Reasons to accept, (3) Reasons to reject, (4) Overall Recommendation, and (5) Reviewer Confidence. You can refer to the ACL "Instructions for Reviewers" pages for more details about how to fill out this section of the review form.
  2. Questions and Additional Feedback for the Author(s): Since we will have an author response process, for questions you would like the author(s) to respond to during the response period, please include them here. This is also the place for you to give suggestions for improvement to the authors.
  3. Confidential Information: Your answers to questions in this section will not be shared with the authors. Here we ask you about the recommended presentation type (oral vs. poster), recommendation for awards, and confidential comments to the area chairs and/or PC chairs.

General Guidelines

Please take a balanced approach when reviewing the papers. On the one hand, we would like to have a solid technical program with high-quality papers describing a complete piece of work; on the other hand, we also want a broad and interesting program, so please keep an open mind when evaluating and recommending the papers assigned to you.

As stated in the call for papers, EMNLP-IJCNLP 2019 welcome submissions focusing on any aspect of empirical NLP. These include, but are not limited to, papers focusing on model design and implementation as well as those focusing on resource construction and evaluation methodologies.

Please note that a short paper is not a shortened long paper. Instead short papers should have a point that can be made in a few pages and present a focused contribution.

Supplementary Materials

Supplementary materials are allowed as a stand-alone document uploaded as a supplementary file. Supplementary materials are supplementary, and you have no obligation to read them. You should treat them like other citations in submissions that may be helpful in understanding background or details beyond the scope of the paper itself.

Secondary Reviewers

As in most previous NLP conferences, you are allowed to solicit help from others. However, when it comes to writing the final review and giving the final scores, we expect you to take the secondary reviewer’s review and re-write it using your own words and adjust the scores when you see fit. Essentially, the final review should reflect your own opinions about the paper, and you need to be able to justify the opinions you present in the final review.

Format of Submissions

The program chairs and area chairs have already identified submissions that violated our formatting guidelines and have desk-rejected those submissions. Therefore, you do not need to worry about formatting issues with the submissions assigned to you.