International Conference on Machine Learning June 26–July 1, 2012 — Edinburgh, Scotland

« back to conditions

ICML 2012 Survey Results

Background

1) Did you serve as an Area chair for ICML 2012?

25/575 = 4% Yes

550/575 = 96% No

2) Did you serve as a Reviewer / PC member for ICML 2012?

198/575 = 34% Yes

377/575 = 66% No

3) Are you an author of a paper submitted to ICML?

466/575 = 81% Yes

109/575 = 19% No

4) You are:

230/463 = 50% An author of an accepted paper

267/463 = 58% An author of a rejected paper

5) Are you going to attend ICML 2012 in Edinburgh?

225/575 = 39% Yes

350/575 = 61% No

Paper Assignment

6) Which information do you support using for assignment of papers to PC members? (choose all that apply)

127/195 = 65% Keywords

158/195 = 81% Toronto matching service

172/195 = 88% Bids

150/195 = 77% Choices made by the Area Chairs

7) How did you like your assignments?

16/197 = 8% Very interesting

168/197 = 85% Good fit for my expertise

12/197 = 6% Mostly not my area

1/197 = 1% Terrible

Paper Reviewing

9) Did you consult supplementary material for any of the papers you reviewed?

128/197 = 65% Yes

69/197 = 35% No

10) Did the supplementary material help you form a decision for any paper?

75/189 = 40% Yes

114/189 = 60% No

11) Did you have sufficient time to prepare your reviews?

160/194 = 82% Yes

34/194 = 18% No

12) Would you prefer if the paper submission deadline was earlier, to allow more time for the initial review period?

31/196 = 16% Yes, the review period should be longer.

151/196 = 77% The review period was fine.

14/196 = 7% No, in fact the review period could be shorter.

Author Response

13) Did you read the author responses for any of the papers you reviewed?

196/198 = 99% Yes

2/198 = 1% No

13_5) Was the author response informative for you?

166/196 = 85% Yes

30/196 = 15% No

14) Did you consult a new version of a paper uploaded during author response?

105/197 = 53% Yes

92/197 = 47% No

15) Did you change any of your (meta-)reviews due to the author response or new version?

47/196 = 24% It changed my mind about a paper.

133/196 = 68% It helped clarify a few questions, but did not change my mind.

16/196 = 8% It was not useful.

0/196 = 0% I did not see it.

16) Do you think the author feedback has an influence on the decision of acceptance/rejection?

144/196 = 73% Yes

52/196 = 27% No

Discussion

17) Did you participate in the discussion for any of the papers you reviewed?

188/196 = 96% Yes

8/196 = 4% No

18) Did you change any of your reviews due to the discussion?

122/195 = 63% Yes

73/195 = 37% No

Assessment of Reviewing Process

19) Do you think the ICML 2012 reviews were different in quality from the reviews at previous ICMLs?

7/197 = 4% ICML 2012 substantially better

43/197 = 22% Somewhat better

67/197 = 34% The same

6/197 = 3% Somewhat worse

1/197 = 1% Substantially worse

73/197 = 37% Don't know

20) Do you think the ICML 2012 reviews were different in quality from the reviews at other similar conferences?

14/197 = 7% ICML 2012 substantially better

68/197 = 35% Somewhat better

75/197 = 38% The same

10/197 = 5% Somewhat worse

2/197 = 1% Substantially worse

28/197 = 14% Don't know

21) Compared to other conferences, how much effort was it to participate in the program committee for ICML 2012?

3/196 = 2% ICML 2012 substantially less

15/196 = 8% Somewhat less

89/196 = 45% The same

48/196 = 24% Somewhat more

20/196 = 10% Substantially more

21/196 = 11% Don't know

22) Throughout the reviewing process, was it always clear to you what needed to be done?

183/195 = 94% Yes

12/195 = 6% No

23) How many hours did you spend on the review process?

Averaged over 161 entries: 28

24) If you submitted a rebuttal during the author response period, do you believe it had some influence on the final decision?

11/455 = 2% Very strong influence

92/455 = 20% Substantial influence

200/455 = 44% Marginal influence

152/455 = 33% No influence

25) Do you prefer having the author response option?

409/462 = 89% Yes

53/462 = 11% No

26) If you uploaded a new file during the author response period, do you believe it had some influence on the final decision?

13/365 = 4% Very strong influence

50/365 = 14% Substantial influence

148/365 = 41% Marginal influence

154/365 = 42% No influence

27) The majority of the reviews of your paper were:

55/459 = 12% Right to the point

276/459 = 60% High quality

80/459 = 17% Low quality

48/459 = 10% The reviewers did not understand my paper.

28) Do you prefer the option to revise the paper during author response?

297/457 = 65% Yes

160/457 = 35% No

29) Do you think that the meta-reviews adequately summarized the reviewers' opinions and made the right decision?

160/459 = 35% Yes

145/459 = 32% Mostly

103/459 = 22% Somewhat

51/459 = 11% No, there was no justification of the decision.

Conference Format

30) With 242 accepted papers, ICML can no longer offer a full talk to all accepted papers (assuming 5 parallel tracks over 3 days). Which of the following do you support (choose all that apply):

134/558 = 24% Accept fewer papers.

198/558 = 35% Add a day.

193/558 = 35% Add an additional track.

216/558 = 39% Present some papers only as posters.

241/558 = 43% Present some papers as short talk + poster.

15/558 = 3% Other (briefly comment below)

30-c) Other

Results (27)

31) The workshop program has been extended to 2-days. What is your opinion of this new format?

151/561 = 27% Good idea, I plan on attending.

196/561 = 35% Good idea, but I won’t attend.

178/561 = 32% I don’t care.

26/561 = 5% Bad idea, ICML is too long already.

10/561 = 2% Bad idea, I don’t like workshops.

32) Which conferences would like to see in co-location with ICML in the future? (choose all that apply):

237/434 = 55% COLT

216/434 = 50% UAI

81/434 = 19% ECML-PKDD

118/434 = 27% SIGKDD

109/434 = 25% IJCAI

122/434 = 28% AAAI

26/434 = 6% ECAI

41/434 = 9% EMNLP-CoNLL

55/434 = 13% SIGIR

56/434 = 13% ACL

26/434 = 6% RSS

104/434 = 24% CVPR

18/434 = 4% RSS

13/434 = 3% Other (comment below)

32-c) Other

Results (23)

33) Feel free to enter any additional comments here:

Results (100)