double-blind-reviewing / double-blind-reviewing.github.io Goto Github PK

Since 2012: https://www.usenix.org/conference/osdi12/call-for-papers

"NEW: The paper review process will be double blind this year."

partial double-blind review introduced with POPL 2012 by Mike Hicks: https://www.cs.umd.edu/~mwh/papers/hicks12popl.html

How the conferences deal with pre-prints is the subject of many discussions in the community. Some (including I) believe that it greatly affects the purity of the double-blind process. It would be great if you can add a column regarding each venue's policy on this subject to the table.

For example in the case of WWW: https://www2020.thewebconf.org/call-for-contributions
".. if available online (e.g., via arXiv) and not anonymous, their titles and abstract must be sufficiently different from the submission to The Web Conference 2020 in order to limit the risk that a direct search breaks the double blind reviewing requirement."

vs. USENIX Security policy: https://www.usenix.org/conference/usenixsecurity20/submission-policies-and-instructions
" While submitted papers must be anonymous, authors may choose to give talks about their work, post a preprint of the paper online, disclose security vulnerabilities to vendors or the public, etc. during the review process."

Partial double-blind as of ASPLOS 2004:
http://www.cs.utexas.edu/users/mckinley/notes/old-blind-2008.html

Not yet sure when full double-blind was instituted (definitely in place for ASPLOS 2020).

Partial ("lightweight") double-blind began in 2014:
https://tupl.cs.tufts.edu/papers/PracticesOfPLDI.pdf

PLDI will commit to use of a lightweight double-blind reviewing, starting in PLDI ’14.
The data from Mike Hicks’
report from POPL ’12 [4] suggests that there is support in
the broadly construed programming language community
for DBR. The primary goal of light DBR is to help PC
members review papers with minimal bias, not to make it
hard for them to discover authorship if they try.

Full double-blind began in 2016:
https://dl.acm.org/action/showFmPdf?doi=10.1145%2F2908080

We employed a double-blind reviewing process; during reviewing, authorship
was only disclosed when deemed absolutely necessary (this happened very rarely), and in
general, authorship was only disclosed for accepted papers. This approach avoids implicit bias and prevents compromising future double blind reviewing for rejected papers, which comprise the majority of submitted papers.

since 2019.

Accordingly, major systems conferences (including SOSP, OSDI, ASPLOS, Eurosys, FAST, NSDI, and USENIX Security) employ a double-blind reviewing process by keeping author identities concealed from reviewers. For ATC ’19, we employed this policy as well

https://www.usenix.org/sites/default/files/atc19_full_proceedings.pdf

From https://eprint.iacr.org/about.html#publication:

The view of IACR and the ePrint archive is that such a posting is permitted and that authors should not be penalized by conference program committees for having made such a posting.

Refers to eprint rather than arxiv, but the point is there. This is somewhat of an informal policy of the IACR covering Crypto/Eurocrypt, although I think in principle PC chairs have the authority to overrule it (which would be unpopular).

Since 2017.
https://www.usenix.org/conference/nsdi17/call-for-papers

"NSDI '17 is double-blind"

https://www.usenix.org/conference/nsdi16/call-for-papers

"NSDI is single-blind"

per Alexandra Meliou

Since 2001.
https://sigmodrecord.org/publications/sigmodRecord/0606/p29-inv-article-madden.pdf
"Starting with the 2001 SIGMOD conference, the
SIGMOD Chair, in consultation with the SIGMOD
Advisory Committee, imposed a double blind rule on
all future SIGMOD conferences."

Full double-blind since 2018: https://htor.inf.ethz.ch/blog/index.php/2018/01/27/sc18s-improved-reviewing-process-call-for-papers-and-comments/

Hi,

Just wanted to point out that FOCS is now using double-blind reviewing: in 2022, in 2023.

Thanks for this great resource!

Ken Birman complains about it for SOSP '05, so at least since 2005.

As SOSP celebrates its 20th Anniversary, we really need to debate some basic policies:

1. Is blind submission beneficial?

https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1095810

In force since 1999: http://www.sigops.org/s/conferences/sosp/1999/CFP/CallForPapers.html

Full double-blind from 2011. Partial double-blind in 2014. Apparently back to full double-blind as of 2017.

https://www.usenix.org/conference/usenixsecurity14/submitting-papers

This year, we are using a hybrid approach to double-blind review. While submissions will remain anonymous during the review process, authors’ names and affiliations will be revealed to the reviewers after the reviews are received, but before the program committee meeting.

The double-blind process was introduced in 2011 to increase reviewing fairness by reducing actual and perceived bias.

https://www.usenix.org/conference/usenixsecurity17/submitting-papers

The review process will be double blind.

I just realized that my PR (#26) did not update the README table.