ICML 2019 Style & Author Instructions

The maximum paper length is 8 pages excluding references and acknowledgements, and 12 pages including references and acknowledgements. The use of 10-point Times font is mandatory.

Please see the LaTeX style files, an example paper, and the Call For Papers.

(Other software than LaTeX is not supported.)

Double-Blind Reviewing

Reviewing for ICML 2019 is double-blind; i.e., reviewers will not know the authors’ identity (and vice versa). Detailed instructions for how to ensure your anonymity are also contained in the above example paper. In brief, please refer to your prior work in the third person wherever possible, and please refrain from adding acknowledgements, grant numbers as well as public github repository links to your submitted paper. If you need to make an anonymous reference (e.g., if you refer to your own work that is under review elsewhere), please upload this work as supplementary material (see below). Note that anonymizing your paper is obligatory, and papers that explicitly or implicitly reveal the authors’ identities may be rejected. A reviewer may be able to deduce the authors’ identities by using external resources, such as technical reports published on the web. The availability of information on the web that may allow reviewers to infer the authors’ identities does not constitute a breach of the double-blind submission policy.

Supplementary Material

ICML 2019 supports the submission of two kinds of supplementary material -- supplementary manuscripts and code/data -- which can be used to provide additional evidence. In particular, if you make anonymous references in the paper, please upload the referenced papers, so that the reviewers can take a quick look. Please also take care not to reveal your identity in the supplementary material.

For code submission, we expect authors to put in reasonable effort to anonymize the submitted code and institution. This means that author names and licenses should be removed. Submission of code through anonymous github repositories is allowed; however, they have to be on a branch that will not be modified after the submission deadline. Please enter the github link in a standalone text file in a submitted zip file. Finally, we understand that many of our authors work with highly sensitive datasets, and are not asking for private data submission.

Note that submitted papers must be entirely self-contained. Reviewers are encouraged, but not obliged to look at the provided supplementary material because we also do not want to burden reviewers unnecessarily. In particular, you must not use supplementary material for extending the length of your paper. If there is material that you believe is critical to the evaluation of your paper, it needs to be included in the paper.

The supplementary material could be submitted as either a zip file or a pdf.

Supplementary material will not be published or archived. If you want to include it in a final version of the paper, you must put it on a web site and reference the site in the final version of your paper.

Reviewer Matching

To aid in finding good reviewers for your paper, we will be receiving help from the Toronto Paper Matching System.

As part of the submission process, you will be asked to provide permission to transfer your paper to the “Toronto Paper Matching service”, which you should accept.

Dual Submission Policy

It is not appropriate to submit papers that are identical (or substantially similar) to versions that have been previously published, or accepted for publication, or that have been submitted in parallel to other conferences. Such submissions violate our dual submission policy.

There are several exceptions to this rule:

1. Submission is permitted of a short version of a paper that has been submitted to a journal, but has not yet been published in that journal. Authors must declare such dual-submissions either through the CMT submission form, or via email to the program chairs (icml2019pc@gmail.com). It is the author’s responsibility to make sure that the journal in question allows dual concurrent submissions to conferences.

2. Submission is permitted for papers presented or to be presented at conferences or workshops without proceedings (e.g., ICML or NeurIPS workshops), or with only abstracts published.

3. Submission is permitted for papers that have previously been made available as a technical report (or similar, e.g., in arXiv). In this case we suggest the authors not cite the report, so as to preserve anonymity.

Finally, note that previously published papers with substantial overlap written by the authors must be cited in such a way so as to preserve author anonymity. Differences relative to these earlier papers must be explained in the text of the submission. For example, (This work develops [our earlier work], which showed that).

Author Feedback

From Mar 9 to Mar 14 (tentatively), authors can see the reviews and respond to their content.

Your response can be used by the Program Committee to alter their judgment of the paper, so it is in your advantage to respond. The maximum length of the response is 5000 characters.

Authors can submit one response per submission. Once author feedback is enabled, authors will see a link to “Author Feedback” from the author console. Any of the authors of a paper can enter/edit the response, and the response can be returned to and edited up to the deadline for author feedback.

Remember that reviewing is double-blind. Do not include any information in the response that can identify you or your co-authors. Please do not include any URLs in your response.

We recommend using your judgement when crafting a response. There is no need to respond to every minor question or suggestion for improvement. Rather, the response is a good opportunity for addressing issues like a reviewer’s uncertainty about a point, a reviewer making an incorrect assumption, or a reviewer misunderstanding a part of the paper. We recommend being polite and professional.

 

Frequently Asked Questions on Code Submission:

This year, we have an option for supplementary code submission that the authors can potentially use to provide additional evidence to bolster their experimental results. Here is a FAQ about our code submission policy.

1. Is code submission mandatory?

No. Code submission is completely optional, and we anticipate that high quality papers whose results are judged by our reviewers to be credible will be accepted to ICML, even if code is not submitted.

2. Does submitted code need to be anonymized?

ICML is a double blind conference, and we expect authors to put in reasonable effort to anonymize the submitted code and institution. This means that author names and licenses should be removed.

Please note that the submitted code will not be made public -- eg, only the reviewers and Area Chair in charge will have access to it during the review period. If the paper gets accepted, we expect the authors to replace the submitted code by a non-anonymized version or link to a public github repository.

 

3. Are anonymous github links allowed?

Yes. However, they have to be on a branch that will not be modified after the submission deadline. Please enter the github link in a standalone text file in a submitted zip file.

 

4. How will the submitted code be used for decision-making?

The submitted code will be used as additional evidence provided by the authors to add more credibility to their results. We anticipate that high quality papers whose results are judged by our reviewers to be credible will be accepted to ICML, even if code is not submitted. However, if something is unclear in the paper, then code, if submitted, will provide an extra chance to the authors to clarify the details.

To encourage code submission, we will also provide increased visibility to papers that submit code.

5. If code is submitted, do you expect it to be published with the rest of the supplementary? Or, could it be withdrawn later?

We expect submitted code to be published with the rest of the supplementary. However, if the paper gets accepted, then the authors will get a chance to update the code before it is published by adding author names, licenses, etc.

6. Do you expect the code to be standalone? For example, what if it is part of a much bigger codebase?

We expect your code to be readable and helpful to reviewers in verifying the credibility of your results. It is possible to do this through code that is not standalone -- for example, with proper documentation.

7. What about pseudocode instead of code? Does that count as code submission?

Yes, we will count detailed pseudocode as code submission as it is helpful to reviewers in validating the credibility of your results.

8. Do you expect authors to submit data?

We understand that many of our authors work with highly sensitive datasets, and are not asking for private data submission. If the dataset used is publicly available, there is no need to provide it. If the dataset is private, then the authors can submit a toy or simulated dataset to illustrate how the code works.

9. Who has access to my code?

Only the reviewers, Area Chair and Senior Area Chair assigned to your paper will have access to your code. We will instruct reviewers, Area Chair and Senior Area Chair to keep the code submissions confidential (just like the paper submissions), and delete all code submissions from their machine at the end of the review cycle. Please note that code submission is also completely optional.

 

10. I would like to revise my code/add code during author feedback. Is this permitted?

Unfortunately, no. But please remember that code submission is entirely optional.