International Conference on Machine Learning June 26–July 1, 2012 — Edinburgh, Scotland

« back to conditions

ICML 2012 Survey Results

Conditioned on questionee being an author

Background

1) Did you serve as an Area chair for ICML 2012?

19/466 = 4% Yes

447/466 = 96% No

2) Did you serve as a Reviewer / PC member for ICML 2012?

89/466 = 19% Yes

377/466 = 81% No

3) Are you an author of a paper submitted to ICML?

466/466 = 100% Yes

0/466 = 0% No

4) You are:

230/463 = 50% An author of an accepted paper

267/463 = 58% An author of a rejected paper

5) Are you going to attend ICML 2012 in Edinburgh?

207/466 = 44% Yes

259/466 = 56% No

Paper Assignment

6) Which information do you support using for assignment of papers to PC members? (choose all that apply)

56/ 88 = 64% Keywords

76/ 88 = 86% Toronto matching service

78/ 88 = 89% Bids

65/ 88 = 74% Choices made by the Area Chairs

7) How did you like your assignments?

9/ 89 = 10% Very interesting

76/ 89 = 85% Good fit for my expertise

3/ 89 = 3% Mostly not my area

1/ 89 = 1% Terrible

Paper Reviewing

9) Did you consult supplementary material for any of the papers you reviewed?

59/ 88 = 67% Yes

29/ 88 = 33% No

10) Did the supplementary material help you form a decision for any paper?

32/ 85 = 38% Yes

53/ 85 = 62% No

11) Did you have sufficient time to prepare your reviews?

75/ 87 = 86% Yes

12/ 87 = 14% No

12) Would you prefer if the paper submission deadline was earlier, to allow more time for the initial review period?

10/ 88 = 11% Yes, the review period should be longer.

74/ 88 = 84% The review period was fine.

4/ 88 = 5% No, in fact the review period could be shorter.

Author Response

13) Did you read the author responses for any of the papers you reviewed?

88/ 89 = 99% Yes

1/ 89 = 1% No

13_5) Was the author response informative for you?

77/ 88 = 88% Yes

11/ 88 = 13% No

14) Did you consult a new version of a paper uploaded during author response?

43/ 89 = 48% Yes

46/ 89 = 52% No

15) Did you change any of your (meta-)reviews due to the author response or new version?

25/ 89 = 28% It changed my mind about a paper.

57/ 89 = 64% It helped clarify a few questions, but did not change my mind.

7/ 89 = 8% It was not useful.

0/ 89 = 0% I did not see it.

16) Do you think the author feedback has an influence on the decision of acceptance/rejection?

67/ 89 = 75% Yes

22/ 89 = 25% No

Discussion

17) Did you participate in the discussion for any of the papers you reviewed?

87/ 89 = 98% Yes

2/ 89 = 2% No

18) Did you change any of your reviews due to the discussion?

57/ 88 = 65% Yes

31/ 88 = 35% No

Assessment of Reviewing Process

19) Do you think the ICML 2012 reviews were different in quality from the reviews at previous ICMLs?

4/ 89 = 4% ICML 2012 substantially better

26/ 89 = 29% Somewhat better

34/ 89 = 38% The same

4/ 89 = 4% Somewhat worse

1/ 89 = 1% Substantially worse

20/ 89 = 22% Don't know

20) Do you think the ICML 2012 reviews were different in quality from the reviews at other similar conferences?

6/ 89 = 7% ICML 2012 substantially better

31/ 89 = 35% Somewhat better

38/ 89 = 43% The same

6/ 89 = 7% Somewhat worse

1/ 89 = 1% Substantially worse

7/ 89 = 8% Don't know

21) Compared to other conferences, how much effort was it to participate in the program committee for ICML 2012?

2/ 89 = 2% ICML 2012 substantially less

2/ 89 = 2% Somewhat less

42/ 89 = 47% The same

23/ 89 = 26% Somewhat more

12/ 89 = 13% Substantially more

8/ 89 = 9% Don't know

22) Throughout the reviewing process, was it always clear to you what needed to be done?

84/ 89 = 94% Yes

5/ 89 = 6% No

23) How many hours did you spend on the review process?

Averaged over 69 entries: 29

24) If you submitted a rebuttal during the author response period, do you believe it had some influence on the final decision?

11/455 = 2% Very strong influence

92/455 = 20% Substantial influence

200/455 = 44% Marginal influence

152/455 = 33% No influence

25) Do you prefer having the author response option?

409/462 = 89% Yes

53/462 = 11% No

26) If you uploaded a new file during the author response period, do you believe it had some influence on the final decision?

13/365 = 4% Very strong influence

50/365 = 14% Substantial influence

148/365 = 41% Marginal influence

154/365 = 42% No influence

27) The majority of the reviews of your paper were:

55/459 = 12% Right to the point

276/459 = 60% High quality

80/459 = 17% Low quality

48/459 = 10% The reviewers did not understand my paper.

28) Do you prefer the option to revise the paper during author response?

297/457 = 65% Yes

160/457 = 35% No

29) Do you think that the meta-reviews adequately summarized the reviewers' opinions and made the right decision?

160/459 = 35% Yes

145/459 = 32% Mostly

103/459 = 22% Somewhat

51/459 = 11% No, there was no justification of the decision.

Conference Format

30) With 242 accepted papers, ICML can no longer offer a full talk to all accepted papers (assuming 5 parallel tracks over 3 days). Which of the following do you support (choose all that apply):

95/453 = 21% Accept fewer papers.

164/453 = 36% Add a day.

156/453 = 34% Add an additional track.

175/453 = 39% Present some papers only as posters.

189/453 = 42% Present some papers as short talk + poster.

12/453 = 3% Other (briefly comment below)

30-c) Other

Results (21)

31) The workshop program has been extended to 2-days. What is your opinion of this new format?

135/454 = 30% Good idea, I plan on attending.

152/454 = 33% Good idea, but I won’t attend.

138/454 = 30% I don’t care.

20/454 = 4% Bad idea, ICML is too long already.

9/454 = 2% Bad idea, I don’t like workshops.

32) Which conferences would like to see in co-location with ICML in the future? (choose all that apply):

190/349 = 54% COLT

168/349 = 48% UAI

72/349 = 21% ECML-PKDD

84/349 = 24% SIGKDD

89/349 = 26% IJCAI

100/349 = 29% AAAI

21/349 = 6% ECAI

32/349 = 9% EMNLP-CoNLL

42/349 = 12% SIGIR

40/349 = 11% ACL

23/349 = 7% RSS

89/349 = 26% CVPR

17/349 = 5% RSS

12/349 = 3% Other (comment below)

32-c) Other

Results (22)

33) Feel free to enter any additional comments here:

Results (83)