Thread by @cHHillee

Some metadata for those curious about their #ICLR2020 reviews.

1. Histogram of the average reviews.
2. Top x% deciles

Seems like reviews this year at @iclr_conf are substantially lower than previous years. Probably an artifact of the new [1,3,6,8] reviewing system. (1/n)

@iclr_conf For experience:
Out of 7583 total #ICLR2020 reviews:
1078 "do not know much about this area"
2484 "have read many papers in this area"
2604 "have published 1 or 2 papers"
1417 "have published in this field for many years"

47% of reviews haven't published in this area!

@iclr_conf For thoroughness:
601 "made a quick assessment of the paper"
4099 "read the paper at least twice and used their best judgement"
2698 "read the paper thoroughly"

(3/n)

@iclr_conf For experimental correctness:
230 "did not assess the experiments"
4485 "assessed the sensibility of the experiments"
2687 "carefully checked the experiments"

(4/n)

@iclr_conf For theoretical correctness:
575 "did not assess the derivations or theory"
3991 "assessed the sensibility of the derivations and theory"
1567 "carefully checked the derivations and theory"

(5/n)