ICML 2020 Virtualization Survey Responses - Results

ICML 2020 announced that the 2020 conference will be held online, due to COVID-19. As part of the shift online, we ran a survey of past ICML participants and current ICML 2020 authors, reviewers and organizers, to try to better understand the needs and constraints of the community. Respondents were allowed to mark if they permitted their responses to be shared anonymously; we have shared those here under the terms of their agreement. The number n in each response is the number of people in this sample (who agreed to share). We hope you find this helpful!

Hal Daumé III and Aarti Singh
ICML 2020 Program Co-Chairs

Q15. Are there any other employment-related aspects that are important to you that we did not list?

finding some potential students to be my master/phd
ICML's purpose is not to be a employment platform, the conference should focus on quality science. The recruiting part is a side-effect.
Meeting potential internship host / team, talking to them.
I like walking around the exhibition area.
Many hiring companies conduct on-site interviews for pre screened candidates. Having a virtual conference would mean this can not be done anymore.
Academia oriented employment (postdoc, assoc. Prof.) is an important option for me
Finding connections for potential faculty positions.
Learn about potential employers that I did not previously know about
Getting to know the research attitude and their research output of big firms
join in some social circles
talking to people who work somewhere to get a feel for what it would be like to work there myself; and to overall build a picture of options fo rmyself
Publicity and outreach for the company (stands, coffee chats etc)
Knowing more in-detail about the job descriptions, which are commonly not very well explained in the websites. It is always better to have a person to person interaction.
Finding excellent graduate students and establishing long-term relationships to help recruit
The list above is found to be complete
Getting to know the teams and ongoing projects better at a prospective employer.
Possibilities for collaborations with industry; consulting opportunities.
Understanding the needs and areas of interest in the job market
Name recognition for my group and my company in the community for purposes of directing future potential candidates to us.
Job fair; and Meeting with prospective employers
The above options are fine with regards to my interests.
no, I'm not looking for a job.
By view of employment case, i do not expect feedback from conference. Of course, there is a low probability meeting someone whose criteria matches with my expectations
Already listed here.
I have tenure so employment is unimportant to me ...
As a sponsor in previous years, we have typically hosted an offsite "recruitment mixer" during the event for potential candidates
Advertizing the positions
Online working opportunity.
To know which companies are looking for researchers in which fields.
Finding out about potential employers, both from them and from people who might be able to judge their quality.
I think the list is sufficient.
Observing companies’ interest/involvement into the topic and the community
making myself and my research more widely known in the community (independent of direct job opportunities)
initiate a self-employment project with others?
Scheduling meetings with members of relevant hiring committees.
Bridging the gap between academia and the industry.
It is important to follow the direction industry is taking when it comes to research and problems. Talking to people at boots and attending demos is a good way of being informed about this.
Discovering new job opportunities
No. Although the conference provides material to turn reality business initiatives.
I am searching for a postdoc position
Finding prospective postdocs or junior faculty. Much more likely to happen at a conference...
Hopefully, I could meet some big company recruiters who look for research scientist.
Discussions with potential research advisor.
Finding a job or internship in which my academic interests are fulfilled.
Find companies and work areas I didn't know yet
Academic jobs (post-docs, tenure tracks)?
Finding funding for my research
As someone who doesn't care much about employment aspects of big conferences (I'm just interested in the research) I wonder if its worth considering having a completely separate event ("job fair") for all the folks (students + companies) interested in the employment aspects.
Finding position at big IT companies.
I really enjoy booths which are not only manned with HR-personal. In fact, those deliver the most natural and honest insight to the working experience offered by some company.
Hands on tutorial session on major research area or platform
Networking for research projects
Finding post-doc opportunities with prospective faculty.
Not really, but here is a precision: Physical conferences are especially important for young PhD graduate who are looking for postdoc positions.
So far, I am not interested employment-related aspects.
Finding a host for sabbatical is currently important
Building connections and finding opportunities (employment/research) to benefit my students/post-docs
Researching the market trend and job openings.
Introduce your idea to potentional investors
Maybe, research before umployment ? It seems with this order that conferences are more for industry than for research purposes
Searching for collaboration opportunities and research intersections with my current job.
Continuous training and critical thinking.
Finding employment for my trainees
get a notion of "benchmark" on what good candidates look like
It can be interesting to talk with contacts that may be of itnerest to my students.
Learning about current/ongoing research at prospective employers
I think with all the recruiters out there and companies aggressively trying to find talents, conferences do not need to consider employment opportunities and mainly focus on their real agenda which is show casing the state of the art research in different areas.
no, I'm not looking for a job.
Seeing what employment options are available generally (what type of jobs, at which locations, etc).
Advertising what our company does! Setting up a booth, etc.
Connections with companies.
I don't think so ... probably not.
what kinds of opportunities are out there and what does companies seek for. Important: Hear these things from the insiders
Perhaps meeting other junior faculty.
Find post-docs and applicants for faculty position.
To have a better experience of the current trends in different areas in industry, so that we can better prepare for jobs.
The most important thing about conferences is just meeting people and getting to know each other. These are the ones who will be colleagues, faculty, or on hiring committees. The standard booth based recruitment is an annoying distraction for conferences.
Finding good talent
Identifying employers who align with my research interests and who are, in general, interested in research.
I enjoy being able to visit the company booths and talking to people there. Not necessarily only to get a job/internship, but to understand what's going on in the industry in general.
looking for a ph.d opportunity
Collaboration opportunities including exchange, visiting scholar, fundings should be included
Networking before applying for faculty positions.
No - I think you got them! :)
Building connections for tenure-track jobs!
looking for companies and universities that can host a phd Student to strenghten potential collaboration
Demos on new technologies by different companies.
Learn about their need.
Please don't make the conference a recruiting event
Finding out about possible jobs for my graduating students. Introducing my students to other researchers (helping them network)
Discovering industry roles for ML researchers
Determining if potential candidates are able to present well & answer questions about their research.
Overall networking, getting as many opportunities as possible.
Looking for post-doctoral and fellowship opportunities
Evolve internally in my company
Supporting my own students in networking with potential employers (arranging contacts and chats)
Post Doc (in last phase of PhD now)
Making connections for potential sabbaticals from university positions
I want to find a suitable postdoc research team.
I'd like to meet researchers from Japan.
I'm quite fascinating for the ones listed already
no, I'm not looking for a job.
Looking for fellowship openings in areas outside of my expertise area
join in some social circles
Understanding the type and range of employment opportunities currently available.
Establishing a strong community around your own research agenda (e.g., joint proposals and applications)
Networking with Professors and advisers and getting an idea about the graduating students in their groups.
Start-up collaboration opportunities reflects another secondary goal.
Brand awareness and B2B relations to establish co-research teams
Building awareness around our company/brand (which is different from recruiting directly during the conference)
Not that I can think of
Finding technical and D&I mentors
Academic jobs (Research Assistant, Post-Docs) would be great to explore
(1). Post-doctorate Positions (2). Faculty Positions
Finding potential internships for my graduate students
Totally disagree with statement that ICML conf is a place for looking for future employment. There a lot of less important industrial conf for data science and data analysis. Th eICML is a place for academic collaboration and forwarding the game-challenge AI technologies

Q16. Are there any other research-related aspects that are important to you that we did not list?

The above options are fine with regards to my interests.
getting online tutorials and training from seminars and workshops
Recognizing research trends, finding applications for theoretical research, getting informal feedback on research ideas.
Teaching and developing courses on research-related topics.
Get good helpful feedback from others on my own research
Having a better grasp of the global research trends
Publication in the proceedings!
Already listed here.
My research development is v important for me
No, but more precisely I think talks are important to get hooked about some topic but we really learn about a topic when reading the corresponding papers in great details.
Being able to discuss ideas with others/brainstorming
It's pretty clear at this point that the paper is more important than the presentation and conference itself.
To hide from my boss
tutorials and workshops
Getting to chat well-known researchers whom I've only known through their papers and not in person
Talking with people in related feels to see if they would be interested in collaborating in the future
Brainstorming meet ups on cross-disciplinary/novel applications
Learn if I can cooperate with other people
Informal discussion of unpublished results and knowledge with others working in similar areas / areas I care about. A few minutes of talking to the author of a paper you thought was interesting can be extremely valuable.
The continuous exposure to talks and posters during conferences usually inspires new ideas. I find this one of the most important aspects of attending conferences for me.
Building network so that people in the same field know me and trust my quality of research.
Networking, discussions, research collaborations.
Open problem sessions. Practical challenges in ML.
discuss with people who got the same inputs from a talk
Hearing how others in the field react to new works that are presented. One of the more valuable things I find at conferences are the frank discussions with people I know in the field about new work from a third party.
Meeting other researchers and finding out about their research best practices.
Learn if I can cooperate with other people
Meeting geographically distant collaborators in person to strengthen the social connection across continents.
meeting leaders in the field and taking advice if my approach is right
Visit the place the conference takes place.
Validating new ideas with others that came up during other talks/tutorials.
getting time away to relax
attending poster sessions many times brings in spikes of new ideas, it's like different shots of light in the sky, it brightens it up. It is different from listening to a couple of talks only.
Learning how to find collaborators
I care about the criticisms, comments and suggestions from the research peers to my own work, it is somewhat important to me.
Possibility to ask for sponsoring deals from companies
Keynotes of AI leaders that significantly affecting the future research directions. Clear separation between new theoretical works and application of known methods in a specific domains
Informal discussions with researchers
Getting feedback and ideas for my own work, even if I am not presenting through chats.
observing what is considered important by the field by seeing which topics/papers are judged as highlights etc
Having more honest face to face talks about research, since people are more willing to talk about downsides and weaknesses of their approaches in private conversation. This helps a lot to get a better picture.
I find panels at ICML particularly useful to learn/discuss big picture questions from/with the leaders in the field. Probably the single biggest distinguishing factor over other smaller conferences and workshops.
The list above is found to be complete
Posters are especially important for me because usually there are a lot of things that I are not clear to me when I read the paper but I can understand by interacting with authors.
Finding some inspired idea from other areas
Learning about new research directions both in relation to my work and to the ML community.
Avenues to run into researchers I would not normally meet.
As a junior PhD student, the ability to meet up with senior researchers / professors who many not have the time to meet me otherwise.
We think Amazon in general is lacking good Science channels for someone to reach out for help and grow professionally. They are cliques of folks who know each other, but we need a way for new comers to form connections.
Tutorials and invited talks discussing future research directions, in areas I'm interested in. (i.e. visionary talks)
Discussing my incomplete work with others.
Sharing my research with the community in a paper.
Interacting with funding agencies and finding out about open possibilities related to my work.
Working with (possibly new) collaborators on research projects (or problems) of common interest.
Virtual workshops and tutorials are key
developing a connection with a researcher that can be valuable in future
Maybe something about taking time to refine/workshop my own research.
I think all are cover
Finding out, which areas are of particular importance in the view of a large group of experts
Having discussions (in the form of panels), about fields, including debates about what problems we sohuld and should not be working on, etc.. Invited talks spark some local discussion, but do not actively engage as much as a panel / some more informal sessions often can.
I am highly interested in attending to keynote speeches, especially if the talk is about potential research directions that are not yet well-developed or are from adjacent fields like neuroscience, biology, physics and more.
That's very much better 👍👍
Access to recorded versions of live sessions to expand the number of sessions to which one can attend.
Impact of physical meeting vs online meetings
Meeting current collaborators and old friends is very important. Forming new collaboration is extremely important.
I don’t have any idea now.
Learning the popular tools, techniques, resources (including data sets) in a research area.
Getting an overview of the research going on in a particular topic area.
With arxiv, I am already quite familiar with the papers published in the main conference, in my narrow area of interest. Workshops and discussions with colleagues are much more valuable for learning about the latest work in my narrow area. For work outside my narrow area, it is globally less important (being outside my area), but the conference is very valuable for me to learn about the latest results.
Get a paper published is certainly a key reason too. Altough I would happily go to conferences only to present a paper that has already been published in a Journal.
Multi domain collaborations.
Learning how to execute Research efficiently
Have talk with other scholars about new noval researching topics which is prospective in future.
To get the idea of in which directions the field is moving forward.
Learning about other research groups working in my research area
I think list is enough for my expectations
Other research events at conferences I would like to see * Panels of experts debating a controversial research topic with time for questions from the audience * Doctoral consortium where PhD students in the early stages of their dissertation can get feedback and general advice
Ask questions which might be useful in extending someones work. Then get them interested to collaborate with you.
Keynote talks from really good people. These can be very valuable (if they are well-done) to senior and junior researchers and to grad students.
Slightly different from above, large conferences are a good way, in my opinion, to get a broader view of a topic than what I can get by reading articles by myself, in particular to see emerging trends, and understand which are currently the most important open questions in the field.
I think the list is sufficient.
Discussing metrics for "good research" in ML (e.g., what constitutes a relevant finding; how can we make sure fundamentally new ideas get disseminated even if they don't *yet* outperform standard benchmarks).
Attending a conference usually frees me from day-to-day work (and family) responsibilities to immerse myself in research ideas. This will be more difficult with a virtual conference. The virtual conferences I’ve attended in the past few weeks have me sitting at a screen for hours which does not seem to inspire creativity or facilitate networking.
get an overview of the field; see how many papers there are in some area (but not in terms of buzz words in the title; rather I mean actually in my mental scheme to represent contents)
I'd like to emphasize that being able to ask questions and interact with others about their research is probably the most useful part of attending the conference. This includes both formally during the presentation, during the poster session, and being able to arrange meetings with other attendees after seeing their paper/poster.
Learning about inter-disciplinary research areas and problems -- how does one use tools from area X to solve problems in area Y.
- On-site meetings with experts to discuss specific research questions with pen and paper ("looking at the math together") - bouncing ideas off of each other in informal settings
Being able to see a lot of work presented with similar topics in a short time frame allows to get a sense of emerging ideas and trends. This coupled with the ability if the presenter to highlight the most important aspect of their work is the main reason for that attending a conference has much more value than simple reading the papers on Arxiv. Another thing is that conferences have a form of 'signaling' where a lot of people will sometimes gather around a poster. This helps filter and direct to works that the community finds interesting. When looking at works outside my field of expertise this helps narrow the field to the most interesting works (granted it is not perfect measure)
Receive feedback from other people on my own research
Not that I can think of!
Identifying the attention of research topics and presenters, and creating a conference speaker list.
Getting a broad sense of what people are working on in my area of specialization.
Discussing my own research ideas with others and getting feedback, potentially avoiding pitfalls early on.
Universal computing power.
As a researcher, research is most important. And in my experience, it is all about meeting the right person with whom the right conversation happens.

Q17. Are there any other networking-related aspects that are important to you that we did not list?

The above options are fine with regards to my interests.
Get to know fellow researchers and potentially initiate common research projects.
Introducing myself to members of relevant hiring committees!
Meeting fellow-students and answer questions about my school
Networking for potential sabbaticals from university positions
Finding ways to get connections with talented professors to apply for be a visitor scholar in their lab
I am interested in _being_ a mentor instead of receiving mentorship.
Helping my students find mentors, collaborators, and employers
Integration of one's research profile link into their account used for virtual conference
Getting meeting one to one with people I am interested to discuss with
I think the list is sufficient.
lunch, dinner together - informal set up discussion - forming teams to volunteer / collaborate
Discovering what are the current hard problems in other research topics and determining how my research could help. Likewise, finding in other areas innovative solutions to my own research problems. Discussing with end-users and industrial partners.
The list above is found to be complete
There's lots of noise in the ML these days, would be good know GPT-2 is not the greatest (no reasoning), for example!
Getting mentorship from experienced/senior researchers in my area
Create bonds within the community
It would be great to have the chance to brainstorm more specifically on how to achieve the kinds of spontaneous interactions implied above. Maybe something like a simulated "lobby" of named avatars where you can "hear" people talking within earshot.
Networking for potential faculty positions.
This ones are better off
Providing mentorship to colleagues who I have not met in a while.
I would also aim to learn about upcoming funding opportunities (e.g., HORIZON-EUROPE calls, etc.).
Providing mentorship
It would be cool to re-use the Whova app like in Stockholm (if possible keep the formed groups around topics). This would avoid searching for contacts made in previous conferences.
I don’t have any idea for that.
Getting feedback from users about the software I work on if they use it in their research.
I generally treat networking as part of "research" as I am not looking for a job or interested in just socializing. I am seeking quality interactions with scientists about science.
May be sessions where different geographies are made to share a lunch table or some other structured forum where different nationalities can interact. Very often people cluster towards their own nationalities.
Learning more about the research directions of more labs, and finding potential collaborators.
I like the socials that connect more experienced researchers with upcoming/budding ones
that point about gossips is irrelevant totally
Listen to other people's questions is always inspiring!
Having casual, unplanned, unstructured experiences, such as meeting people while walking around the booth area and getting drinks, or having a group happy hour.
Application and contribution to human society.
It covers almost all i thought about
Learning about "negative results". Things people try and don't work.
The casual conversations in hallways and lunches not poster session are the most important. I worry switching to virtual may lose this aspect.
Business relationships
Research collaboration and potential Employment networking
Already listed here.

Q18. Is there anything else we should know about scheduling and timing that's relevant for you that we didn't ask about? In particular, are there things we could do to make it easier for you to attend?

Timing is very important
It is difficult to predict the time requests for July.
Make sure that recordings of talks are available immediately after the talk (ideally it should be possible to watch an ongoing talk from the beginning).
I'm a postdoc, so I'm pretty flexible.
Please keep in mind that not only would the attendees be at home, but likely so would their families (kids especially). It will simply not be possible to attend a virtual conference in the same intensive, non-stop way that we attend an in-person conference. I sort of made up some numbers for questions 11 and 12, but I could imagine several configurations that could work. I could probably arrange intensive attention for 1-2 days, medium attention for a bit longer, and low attention (an hour or two a day) for arbitrary amounts of time.
I am not sure if this is a timing suggestion: it might be good to have pre-recorded talks/presentations (even for posters) that could be watched at any time and only have interactive Q&A sessions. With such setup, shorter time periods would be needed for strictly allocated timing.
Please make sure there are enough breaks
Wherever active interaction is required, choose timings which are comfortable across major timezones ex. 5-9 pm (UTC+2 i.e. EU) allows people on both east and west coast to attend.
How many days is correlated to how many hours I would attend: e.g. I think I would be willing to attend 3 concentrated days (e.g. at 8 hours per day), 4 days at about 6 hours per day, 5 or more days at 4 hours, and so on.
You should probably ask people whether they would actually want the conference to be held digitally, or whether it would be better to just cancel it. I, for one, will not be "attending" a digital event. I realize that you guys want to maintain the community, but for me there is no community from a distance. I like to be with people and have spontaneous interactions and meet new people, and none of that will be happening from a distance.
I don't think you factored in daylight savings time. I answered assuming that you will correct that and adjust the standard times given on theform to dayling savings times.
Visa conditions and travel airlines
It would be more important for live interactive sessions to be held at times that we are available in, and less important for sessions that present pre-recorded material to be held at those times.
Some gaps between the sessions, such that people can get other important stuff done as well. Blocking large chunks of time makes people optimize smaller chunks out of them.
Having a structure for the week such that workshops are on specific days, etc.
Thanks for reaching out to everyone rather than just making assumptions!
I would be willing to watch pre-recorded talks at almost any time. Having discussion during more convenient hours would be more useful. Having the ability to pre-network (reach out to people interested in a given workshop or tutorial) and make connections before the start of the conference would be very helpful- then we can try to coordinate closer to the actual conference.
I interpreted questions 9 and 10 to be in terms of the time zone entered in question 8.
Already listed here.
Consider having poster sessions split into several time slots. That way one could check in at whatever time is convenient. My time estimates above assume that I will be locked up with my children not in childcare, which is the current situation for many of us -- in this situation, flexibility is extremely valuable.
Ensure enough short breaks, full time concentrating on virtual presentations can be stressful.
Not sure if this is a good idea but is it possible to use a forum for the poster section? Presenters post their posters there in separate threads and people can discuss / ask questions / make suggestions in replies. In this way everyone can attend when it is most convenient to them.
Be sure that there will be some "pauses" and not a continuous flow of presentations. Avoid too much parallelism between sessions, but maybe everything will be recorded and available offline, so that this may not be a real problem.
I hope there's a way for people to agree to present in multiple times, even if they're on different days. For instance, they could be available 8-10am on one day and 4-6pm on a different day to discuss their work and answer questions.
It would be nice to have videos ready for replay within a few minutes, and also a way to send questions throughout the conference in general (so they can be answered by authors asynchronously).
The most important would be a clear schedule and it would help to be able to replay missed talks.
It's better to have recordings of the talks/poster session.
I would like be paid for the time I will be attending the conference cause in my Country we pay for internet and the Campus is closed
The more recordings made available, the better.
I can't provide a meaningful answer to "11. How many days of the conference would you likely want to (and be able to) attend?" until I know what the schedule is like.
Allow for replays of the talks
If all the talks are recorded, I am planning to play around with live vs recorded talks everyday to watch as many as possible.
If it is possible to record sessions, that would be excellent. (With that, I would strongly recommend being clear about which sessions are recorded and which are not, even as the sessions are happening). One of the big reasons that I attend conferences is for the "hallway track" where you can informally talk to people that you don't have the opportunity to see usually. I am sure that this is the hardest part of an in-person conference to replicate/substitute. If/as you think about possible options, I would love to see explicit programming that connects more senior researchers with early career researchers and connects marginalized researchers with senior researchers. (Again recognizing the inherent difficulty here.)
reduce the registration fees. allow for pre-recordings.
Options to send questions ahead of time for Q&A sessions after talks, which should be recorded.
I would very much prefer to vote on whether I present in the morning or evening for those cases, where it doesn't make much of a difference (even if such a slot will not be possible)
Since I will be at home, my usual family constraints (taking care of babies, picking up in daycare, etc) will apply. Thus, significantly less time for the conference than if I were to physically attend.
Friday and Saturday are religious weekend in Israel. The attendance is impossible
The biggest question is whether or not my children will have childcare in July. It is very difficult to do things with them at home.
For things such as virtual poster presentations, it would be good to define slots where you can book 15 minutes with max 5 visitors plus the presenter, so everyone has a chance to ask a question.
Upload video presentations, so anyone can download them.
It would be great if pre-recorded sessions are available for us to attend at a convenient time.
It is harder to pay attention to talks during "somewhat convenient" or "inconvenient" hours, because you're too tired or too sleepy, so I typically don't even bother to try. Whereas the discussion or questions are easier to follow at almost any time, especially with some prior familiarity with the subject. Maybe if the prerecorded talks were released a week or so in advance for people to watch at their preferred time and pace, attending live sessions at too early/too late hours would be a better use of time.
Please record everything and make it accessible so that if someone cannot attend a particular time (or if kids at home get in the way) they can still watch later.
It would be helpful to be able to stream talks in which I don't feel the need to ask questions at a later time.
To avoid burn-out, it might be interesting to try have a gap -- eg, 2 days of material, then 1 day break, then 2 more days, then ... [Rather than contiguous meetings]
post the schedule earlier
Video recordings that can be played back at different speeds, to listen to talks at a faster pace.
Please release the schedule as early as possible so that I can make my arrangements.
make it single-track and spread it out over weeks
Observant Jews wouldn't be able attend between Friday night and Saturday night
Allowing sufficient breaks between sessions for getting refreshments or answering crucial emails would be useful, even though the sessions are virtual.
It would be excellent if talks (especially presentations and keynotes) can be recorded and made available to watch offline.
If we are clear on what parts of the conference will be live vs asynchronous (i.e. recorded talks, Q&A sessions, breakout sessions, etc.)
Well planned session would be preferred.
Announce the meeting date as soon as possible so that people can adjust their work arrangements.
Is that possible to replay the talks?
To let participants of all time zones see all presentations, they'll need to be recorded so they can be watched afterwards, and some online system should allow to ask questions to the authors and/or discuss in a larger group about the work afterwards.
i didnt know how many days the conference will take so i will be available all days
I propose the following idea: 1- All presentation to be recorded so that depending on the different time zones, people will be able to attend very easily. 2- Furthermore, it would be great if the presentations was available before the talk time lets say (24 or 48 hours before). The reason for that, is within that time slot (48 hours) people will watch the presentation and be able to ask all possible questions and then in the time allocated for the presentation, the presenter will present the presentation and after that he will answer all questions received from all people and this will make it possible to answer so many more questions and from different time zones and also this will be recorded so everyone will be able to watch it based on their time zone. I will more than happy to provide any help or volunteer in helping and organizing the ICML virtual conference. Thanks a lot and I wish you all the best and stay safe.
Include breaks because we cannot attend with high concentration for a long time in front of our computer, without interactions
Connecting during the week-ends from home is very difficult for parents of young children (say 0-5)
Scheduling should be announced as soon as possible.
I understand it's not possible to meet everyone's constraint, and appreciate your making this effort. Thank you. Besides, for my own presentation I would want to go when the audience is the largest, even if it's at a time I have indicated is not convenient, and I perhaps am not the only one. So the attendance timings possibly matter more than presentation-time preferences (certainly true for me).
Maybe have every talk being done twice? This could give an opportunity for each set of two overlaping time-areas (Americas, Europe, Asia) to attend the talk live once and ask questions.
Availability of lectures offline
In the schedule, lunch-time should be considered as well. 10-12 or 14-16 is very convenient. but 12-14 is not, as I need to handle/cook lunch anyway (though 1 hour would be enough). However, only 10-14 or 12-16 options are available :/
Given the remote format, it would be easier to participate if the conference were spread out over more days (with fewer hours of conference material each day).
I will not be available Wednesday evening (after 4pm) , Thursday evening (after 3pm) and all day Friday
I think this global timezone effort is making Amazonian life miserable on the average. Please stick to Seattle time for this conference, we already have enough timezone pain as part of daily work!
I don’t have any idea.
If possible, it would be very convenient to have the recorded sessions available after the presentations as well so that attendees who may miss the sessions will be able to access. This would also make it easier to plan the schedule about which live sessions to attend and which sessions to view later for maximum information absorption.
post the schedule earlier
can we go decentralized? e.g. put up online talks at time X, and 10 hours later there is the Q+A. Then everyone can watch the talk within the time window of 10 hours...
based on number of interest in each timezones, the slots can be divided with max no attendees
No. Due to the COVID-19, I have a lot of free time to attend the meeting. Thanks.
I am working from home, and it's likely to be the case during the conference. My answers would probably change if I had to work. For now, I dont.
Postpone the conference and make it physical!!
I know it is hard for organizers to accommodate all everyone's request. I will be flexible with the timings and plan other activities around my attendance of ICML.
Being located in Europe, I would of course prefer if a lot of the content was available at 'regular' time zones. Nevertheless, I understand the point that this is logistically impossible.
Tuesday from 2pm to 5pm for class lectures.
No, this is ok to my opion
Recording all talks/presentations would be a massive help!
It would be nice if at a virtual conference the switching between sessions would be easier, i.e. not demanding to switch the Zoom space.
just keep all formal research content as flexible (i.e. non-live, recorded) as possible
I'm on lockdown so everyday is the same for me.
If live sessions are recorded, that would help (even more so if sessions happen at the same time).
Perhaps have the sessions recorded for future viewing. I understand it will not be possible for you to hold sessions at a time that would be convenient for everyone. However, if the sessions are recorded, people might be interested in listening certain talks later when it would be more convenient for them.
Anything recorded is much better than live sessions. Allows to watch at >1x speed (VERY IMPORTANT), skip uninteresting parts, and pause/resume to fit around our other activities.
Monday and Friday will also work except between 10:00 and 14:00
I will not come to a virtual conference I will inly attend real live ones
It would be great if talks are recorded (I guess that's already the plan) and a second Q&A session would be offered to accommodate people in different time zones.
Would it be possible to record talks and leave them up for a week or two after the conference so we can watch them at different times. Being from Australia, I suspect whatever times the conference will be on, will be at a weird hour for me. Being able to download PDFs of posters would also be great.
I am rather flexible with my time. The most important aspect for me is a clear organization that allows me to plan for it.
If you record the talks, then people can watch them whenever they wake up or are free. If you have parallel sessions, you can actually accept all the good papers instead of rejecting them for ridiculous reasons too!
- sharing presentation slides\materials in advance for every session
Would be better to stretch the conference out over a longer period and have it be fewer hours per day
I think a key point is to make the exact schedule early on and easy to understand. This way I can plan carefully my days in advance, to make them fit with the conference events that I want to attend.
My main focus is to 1) attend as many research tutorials/talks as possible to be learn as much as possible 2) talk to potential collaborators 3) network and talk to potential contacts for future employment opportunities
For all that I would love to attend, I'm very unlikely to make it.

Q19. Are there other technologies (or styles of technology) not listed above that you think we should consider?

Microsoft Teams Youtube
Perhaps something like a "YouTube Premiere" could be considered as another possible solution, if not already considered.
Pre-recorded videos over streaming / download-only platforms? E.g. TED-talk style videos with more graphics.
Recorded talks are worthless in terms of "virtual conference", people could just put them on YouTube and you make a playlist. You need to arrange questions and discovery nicely. Chat rooms etc are often more convenient for that.
Generally, I would not like ICML promoting commercial technology if there are good free an open alternatives (e.g. I would prefer mattermost over slack).
Private message boards?
Google Hangouts Meet
TecentMeeting and DingTalk are the most used conference tools during COVID-19 for us. There are also a lot of talks are lived in platforms like Bilibili and Douyu, where audience can send words in real time.
Virtual whiteboards (e.g., Google Jamboard)
Minecraft poster session would be epic!
Some kind of chat/message board, especially around the workshops
My only suggestion here would be as inclusive as it can be and take into consideration disabled people, for example, it would be nice (though I understand it might be hard) to have supertitles/ closed captioning during talks (for people with hearing impairment)
Facebook Live has worked well for streaming of videos and they have scaled up their infra to support lots of real time comments for q&a etc.
https://screencast-o-matic.com
Recording videos with own software rather than Slideslive (makes it much easier because one can cut it rather than having to start over after every mistake).
Discord. But that is probably more for Gaming.
I can imagine many presenters will have a hard time recording presentations at home, without assistance from their respective institution. It would be very important to publish clear guidelines with best practices (tools, etc). If there is any bandwidth say for reviewers to help also reviewing these presentations and help making them better, I'd connect folks in that way. I did this last NeurIPS for an oral, but this time, I suppose you'll need such videos for every single poster.
Google Hangout (falls under video/audio conferencing)
Please have us submit back-up Videos of our presentations, to ensure low connectivity does not affect the audiences experience.
Maybe posting videos on Youtube. They have one of the best video player
it would be great to have recorded videos available soon after the actual talks.
Workplace by Facebook
Please consider privacy-respecting softwares
There are enough options.
Discord might be an option since it has simple voice channels.
Anything will be fine. It's not like this audience knows nothing about computers.
Reddit AMA/similar with authors after presentations
We should use a video channel (yotube, vimeo) to keep the recorded talks for a while. If the conference is virtual, I expect to be able to watch/listen after the conference.
Maybe questions and discussion of presented works could be significantly extended to continue in parallel with the main program for interested participants
There are also similar technologies like Google meet and Hangouts.
OpenReview-style discussion threads for papers
Discord groups could be nice way to socialise
As I suggested in a previous question: forum. Make (at least part of the conference) asynchronous so that people in different time zones can attend when most convenient.
No, but consider that I (at the present moment) have an old computer that is not able to process videoconference easily (it causes the computer to overheat); also I have a low bandwith internet connection. More generally, consider that people may have specific machine limitations.
OBS Studio to record talks
There are too many technologies already. Simple things such as Zoom are preferred.
broadcast the live presentations in a TV program / or multiple TV programs
Long-term video repos (youtube / video proceedings of the conference, for posterity). I know this clashes with the idea of a conference as an event local in space in time, but it's an advantage of the virtual format that presentations can be more easily kept for posterity.
More multimedia / interactive pre-recorded presentation methods like videos with demo pauses or similar
Google Meet, Microsoft Teams, Discord
I don’ have any for now.
Crowdcast(https://www.crowdcast.io/), it allows folks to chat and ask questions (dedicated section) while watching the stream. Sort of like Twitch adapted for conferences. The Neuromatch conference (https://neuromatch.io/) used it recently.
Microsoft Teams over Zoom.
Although I marked many of the proposed systems as "I don't know what this is", I'm eager to get up to speed with them if the conference organizers think it would be appropriate.
IRC, discord :) Jitsi (opensource video calls)
Team Viewer, Discord, Skype
There is a lot of VR that is unexplored and that can be used for conferences beyond the small room style, you can take inspiration from MMOs (massive multilayer online games).
Already listed here.
Zoom and Slack are not stable in China. Do you consider some other apps such as Tencent Conference or WeChat?
I really like confex, which is used by the Ecological Society of America for their attendees. It has nice cross-device scheduling support, conflict resolution, etc. Heaps better than whova!
I hope Chime improves in the long term (they don't even have delete or edit functionality yet!), but please use Zoom for this conference.
Zoom and WebEx are banned from my institution. They are also quite average. Live video streaming and a separate text system (slack, discord etc) are quite good for a conference with dozens to hundreds of attendees and a single speaker.
In the past few days I was actively reading about open-source video conferencing solutions and user opinions/reviews. Note: Closed-source tools are by definition typically rather buggy and have privacy issues (such as Zoom et al.). On the open-source front I found Jitsi Meet (https://jitsi.org/jitsi-meet/) interesting: good functionality, decent encryption, easy usage. Their Jitsi Meet Server is also open source (https://github.com/jitsi/jitsi-meet). We are just testing it locally to replace Zoom. => ICML could be interested in setting up and deploying own Jitsi Meet Servers. I would be happy to hear about your experiences related to question-15, particularly about open-source tools.
Tencent Conferencing
Twitter/ Social networks (eg following a hashtag)
Microsoft Teams (Business version of Skype?)
Social media (such as Twitter), message boards (such as reddit), etc.
Virtual environments a-la online gaming for networking
I find WebEx less convenient than Zoom
When I'm attending virtual talks I want to see the face of the speaker and the slides at the same time. As a speaker I want the audience to see my face and my slides. As a speaker I want to see people who are listening.
Pigeonhole Live is an alternative to Slido
Not that I am aware of
I believe VR would be the best technology to create a fully immersive experience in the long run (read: unsure whether it's doable "on the fly") - there are a few extra costs for participants (e.g., VR helmet) and, depending on the VR experience, it can be quite confusing. As much as I love physical attendance for all the reasons I outlined in this survey, I feel VR is a promising technology we should - as a community - push and explore. But we need to get this right.
Algorithms to match people according to their research/interests. See mind matching at https://ccneuro.org/2020 or https://neuromatch.io
Other VR options, HTC vive, oculus compatible, ICML VR hangout and social rooms
Twitter like social media, reddit like boards
I'm using ShowMe (https://www.showme.com/) for my course and I really like it. I use it both to record lectures (as a whiteboard and/or slides with audio) and "in class" hooked into Zoom as a live whiteboard.
Record all presentations and make them available immediately so people can watch at their own convenience.
Please make sure to prefer tools which have good data protection over tools which run more stable. E.g. please do not use closed source tools like Skype, Zoom, Microsoft Teams, Slack etc. Research should not be compromised by non-transparent software.

Q20. Is there anything else that we should know about your connectivity and/or device situation that would be useful for us in planning? Are there things we didn't ask about that are important. How can...

There are also a lot of talks are lived in platforms like Bilibili and Douyu, where audience can send words in real time.
Wifi is never completely stable here... In many cases we resort on audio + screen only during Zoom teaching.
Does VR imply special hardware?
Connectivity isn't so much an issue as the amount of time I am available with a quiet background.
Planning to attend the conf from university. Some security restrictions (e.g., access to specific servers for demos, etc) could be applied
In the best interests of everyone and speaking in general, try to have people use less bandwidth by switching off videos as much as possible. During presentations/tutorials have the speaker only turn on the video. Try to use slack as much as possible. Try to use Youtube live streaming and its live chat option for questions from the audience and try to address them as much as possible via volunteers watching the chat feed. Share presentation materials beforehand so that people have them handy rather than downloading them then and there during the session. Request authors/presenters to have a high quality video as much as possible and focus camera on white board/slides etc. rather than on the speaker. Do not shaky/noisy recordings as much as possible. Request authors/presenters to use a room which is free from distractions/interferences as much as possible.
Being able to watch videos offline would be great. Not only because of the time zone issue, but also because in a normal conference there are a lot of parallel sessions.
It is hoped that multiple channels can be provided to download conference videos.
I suspect that one of the best ideas will be to have recorded talks and live Q&A. Also, Q&A should be moderated in some way (even if it is control by the speaker), so that you don't have trolls (intentional or unintentional) that impair the experience for everyone else.
Even though I expect that online presentations will work most of the time, I will be very surprised if some will not fail due to fluctuations in network connectivity. For those cases, it may be useful to have an offline plan (e.x. prerecorded video can be launched).
Make both P2P and central servers available, use central server as torrent seedbox
By "video streaming" in the above, I mean ofline streaming (e.g. youtube, etc) not direct, real-time streaming. I can probably pre-record ad download video/audio. Not sure I can live stream anything.
Depending on the pandemic situation, we might be able to organize a local "meet-up" to jointly watch talks and exchange with local ML researchers.
It all looks good. Thanks.
If the network load is high e. g. between 09am-4pm the connection may not be always reliable
streaming + recording on youtube
With a small baby, having recorded sessions (even if they are available only for a short time) would be useful since sometimes I am needed unexpectedly.
It would be great to test the viability of a solution before hand. For instance, one introductionary talk or forum could be given ahead of time to see if everything works correctly given the scale.
Please consider the different people devices e.g. potential compatibility problems with mac or Unix-like systems
NeurIPS "slideslive" is the best conference software I have seen. I watched live remotely and reviewed offline. https://slideslive.com/neurips
It may be helpful to have a "virtual whiteboard" interface, where participants can write/draw using a pen or similar, especially considering the symbol- and visual-heaviness of many ICML papers.
It should be easy to see/search/chat with who else is attending the conference. This should be much easier than e.g. in the Whova app. I expect to be using a desktop browser. Perhaps Google Hangouts or Facebook Events or so.
EVERYTHING should be recorded (with presenter permissions) so it can be watched at a later date, in case guests lose internet connectivity etc. And please, don't post the videos weeks after the event - it should be immediately available.
Make everything as low-bandwith as is possible.
Making talks and videos available for download with a good interface would be a big bonus.
I do not have a VR device to participate in a VR experience
It would be awesome if we could stream previously-recorded videos, instead of need to watch them live.
I expect the above, but it can be intermittent.
Caching streamed videos would be nice, that way you could "skip back in time" in case your connection is lost without missing parts of the presentations
VR headsets are currently largely sold out, so for many people it might be nigh impossible to attend VR meetups.
I live in China and it is very difficult (although not completely impossible) to watch videos in youtube and twitch. There are some established methods that can be used to avoid censoring of the Great Fire Wall here, but whether it will work at a specific time is somewhat luck dependent (e.g. if there is going to be an important political conference near that day, then a large portion of above mentioned methods will not work consistently).
It would be better if there were some later subtitles.
Not that I can think of!
Please you can support with Softwares needed to connect
I am on a 4G connection, somewhat reliable, but not 100%
I think a prerecorded presentation with the speaker being available for live questions would be perfect. Live presentation is not necessary.
First, I would like to thank you, for many years we've been in need for new ways to organize conferences, I think this will be an amazing experiment and we all will learn a lot. I have one suggestion: Each conference has a lot of different presentations, posters, etc. In apps like whova, or even in the paper version, the biggest challenge is to focus on the important presentations out of a magnitude of less-interesting topics. Every day, I have to go through the whole list again and again, especially as the schedule changes and things are added. At NeurIPS, it wasn't effectively possible to add posters to the schedule, so I had to review the whole list again and again. Also, I had to dig through hundreds of communities and events to check for interesting new events again and again. Many of the meetups are very specific (e.g. graduates of xyz), so it would be great to be able to ignore those permanently. Would it be possible to implement some mechanism to filter contributions as a first step? If we could tag every event with - important/maybe/ignore and then have the important and maybe ones show up on a schedule while the ignore ones are suppressed. That would make it much easier to go back and forth between schedule and program and refine the selection.
Video vs audio only communication have very different bandwidth requirements. At work we start with video where everyone greets everyone else and then switch to audio only, which is seamlessly done over Microsoft Teams. Any system we decide to use for long distance presentation should enable this kind of switching behavior.
There may be bugs for real time discussions. I dont know if there's a way to avoid it or to mitigate the waiting if this happens. Anyway, i think there should be some alternative in case of bugs e.g. when the speaker has no connection
Increase the rate of acceptance, so that more ideas can be shared in this unique online ICML conference without discrimination around the world. Previously only attenders can experience the presentations from the author. Now the whole world can share it.
Bandwith is OK (e.g. for downloading larger files) but there are regular short interruptions, which make live interaction (video/audio) difficult
During out-of-office hours, the limiting planning factor is not the device per se, but the child care situation. In a small flat, it's not easy to explain a toddler that daddy needs some time alone right now.
Germany has around 50MBit/s in average locations and due to Corona it might be less.
I live in Iran and, as you might know, the internet is censored here. For sites like youtube, twitch or facebook I will need to use proxies to be able to connect which would degrade the performance considerably.
There will probably be blips in connectivity, so the word "expect" here should be taken in the probabilistic sense.
clear instructions for the technical side and the possibility to try it out beforehand
Whether I can participate in a VR experience might be strongly dependent on the time of the day during which said activity takes place. Speaking live would also be considerably more inconvenient during the night. At the same time, my ability to record a video of myself (or to stream live) is highly dependent on the quality of video production that you expect; I have easy access to a webcam on a tablet, but a more "professional" setup is very likely out of reach for me.
If am given money for internet I could easily be available online and my schedule does time my only problem will unlimited paid for internet
Option to choose video quality should be there.
My above expectation and experience is based on public domain, however, there will be an impact (lag) to live streaming through a VPN or private domain.
Home internet varies between "fast and can reliably stream video" and "slow and video not possible"
Something network in my area resident is very fustrating
While I said that "I expect to be able to stream video of myself", I say this with 95% confidence, meaning that 95% of the stream would be perfect, but there may be a 5% where it is not. There seems to be some bandwidth issues now that everyone is at home.
Very low internet bandwidth owing to so many people working from home
Allow watching videos at >1x speed
I can speak and do video live if I tether to my phone. But my data plan isn't sufficient to allow me to do the whole meeting that way. I would switch to my cable internet, which occasionally has audio dropouts and video freezes.
Already listed here.
Although I can watch streamed video, currently due to the high load, many of streaming services are experiencing problems. Lags and disturbances in live streaming usually cut attention making it hard to follow and re-focus again. I would avoid streaming if the bandwidth cannot be insured.
The microphone and video quality depends strongly on the device used. I do not have professional equipment, only my laptop with the standard equipment. This could be annoying for the audience.
The wireless can go in and out. So most of the time all of 17. will work but sometimes it is a disaster.
Testing these things ahead of time would be nice if possible. I am technologically inept in many ways, an add thing for an ML researcher.
Depending on the COVID situation, I may have easy access to a slow-ish connection (home) and sporadic access to a very fast connection (work)
Ability to record sessions I cannot attend because of scheduling conflicts or odd timing constraints for later download could be very valuable to me.
I have high speed internet, and it is very stable. I am more worried about the tools (Question 15) and the logistics for a smooth experience for everyone.
The above questions are not sharp. If I can stream videos, then can't I listen to audio ?
"real-time chat" doesn't explicitly state if it's textual chat. I expect there to be sporadic interruptions in streamed video (uploading/downloading) if the quality is high. Would be nice to have options that take into account video quality.
I might have limited upload bandwidth if we're still quarantined. But the idea of livestreaming on twitch seems the most appealing to me.
My connectivity is rather poor; 10 mbps here at home. Using my phone as router I can get higher speeds, but the connection is expected to be unreliable, e.g., it might have lapses of 10-15 seconds, although it's unlikely it will drop completely. Although I marked the boxes "able to speak and watch video", my connection is not expected to be perfect.
Not for me, but some people have 32bit or just smart phone access only
I would love to see as many VR options as possible.
Streaming would be nice, but not necessary. It might be difficult to ensure a smooth experience. I think the existing plan of a prerecorded presentation, and live Q&A makes a lot fo sense. For low bandwidth participants, Q&A can be done with text chat.
I believe that content that is a one-way communication (e.g. talk, "poster video") should be pre-recorded. The only live sessions should be Q&A and chats.
It would make sense to formulate a standard procedure for the Q&As, where the speaker briefly repeats the question to potentially make up for a connection loss
I don’t have any for now.
If you rely on open, free tools for all the video conferencing, it would be great if there could be some Plan B in case this is getting overloaded. It would be unwise to expect these tools to just work at scale all the time, for every attendee. Even better if one of your big corporate sponsors could jump in and help to make the technology work.
It would be nice if needed software is linux/android compatible.

Q21. Based on your past experience, what has worked well (in terms of logistics, tools, scheduling, and anything else you think is relevant)?

It was very smooth. I was the remote speaker during my session
I managed to give my talk, but I did not listen to other talks because of time zone difference and childcare obligations
https://www.crowdcast.io/ Mind matching
we were using the Tencent video conference it seems ok, the latency is low.
It was somehow strange but the recent events have made virtual contacts and conferencs a much more known animal so my experience today would be probably much different.
it was short and spread over few days
actually it requires planning and ensuring that am well connected to an high speed network, i have the speakers and well nothing else
As a remote participant it worked well.
easy to choose relevant talks/workshops/tutorials
Online audience being able to talk amongst themselves in text chat
Live-streaming of talks via Twitch, panel sessions via Zoom.
Zoom works great. Chime works as well
Zoom was ok. I ma guessing they were using webinar version of Zoom where an MC was organizing and giving permission to folks to ask questions and mange the talks. Its less reliable than an already recorded video but assuming that all the speakers are fairly well trained with Zoom it should run smoothly.
Slack channels for discussion, posters along with zoom for the lectures and QA.
slides with embedded video delivered ahead of time web page to submit all the remote presentation material
The www.crowdcast.io website used for broadcasting talks worked very well, as did the ability to answer questions that were most popular (some speakers did that as the questions were asked!), and the asynchronous communication.
we were using the Tencent video conference it seems ok, the latency is low.
Recording your presentation and then attending live QA
slido, video streaming
It was really good, Slack channels and youtube and online Zoom conference were used to provide a positive experience.
Having parallel streams, having flexibility to move between streams/rooms, sending video links afterwards, providing presentation materials even afterwards
Having moderators and having things in order from talk to talk (so as little wait time as possible in scheduling, but also scheduled breaks). Also, not having too long of a conference (eg 2+ weeks is too long, people get distracted with life)
Slideslive for recording 5-minute video abstracts worked reasonably well but I would have preferred recording it with software of my own choice.
Logistics, tools and Scheduling
Poster presentation on Slack (with optional short videos to present the poster) Questions from talk on Slack (with selection of questions by the area chair) and written discussion after the talk Zoom webinar Videos available after the talk
Crowdcast was great. Youtube stream was also good.
tools ok: zoom + youtube recording; schedule ok (everyone was very concise; speaking a bit too fast; finishing ontime or before) logistics was ok for almost everybody.
Livestreams of talks
voice quality can still be unbelievably bad
There were almost no difficulity to listent to presentations with zoom.
1 on 1 networking with grip (basically, an online matchmaking service)
Zoom video conferencing worked well.
Moderator taking care for all tech problems is MANDATORY
Zoom worked pretty well.
Having a large percentage of talks available to be vieed at any time.
Video presentations, presenters who know how to use the technology
We used Zoom and Slack with all presenter times in the conference venue time zone, there were pre-recorded videos on the day as a back-up if anyone was experiencing connectivity issues. The size of the conference made this possible, I imagine that due to the size of ICML the logistics might require smarter use of these services, such as only allowing a certain number into zoom rooms (just like the physical conference) to prevent their server bogging down.
Slack channels, videos worked well
I think it was easy to switch between talks and in general, it was a nice experience.
Zoom was used at the conference. I, as a participant, was satisfied with this virtual environment, especially in that I can clearly and easily listen to poster presentations anytime. I heard that many participants said similar thoughts. However, I also heard that some presenters found it uncomfortable because they couldn't see the audience's reactions or faces.
we were using the Tencent video conference it seems ok, the latency is low.
Invited talks, research talks and poster talks mostly worked fine. IMO pre-recorded talks are useful even if people don't find time to listen to them. Overall, IMO, the technical side is as good in virtual conference as in physical conference.
Scheduling ahead and being able to participate in real-time discussions
Crowdcast worked well
- Local. time-based extra programs & events - Recorded session to review later.
it was very simple. Most things went pretty well.
give presenters ipads and apple pencil so they can share their screen and it looks like a whiteboard
live stream and recording simultaneously. linking slide presented to time in video and vice versa.
It worked pretty well for my talk. A person hosts the Zoom session and allowed speakers in order and collected questions.
EDBT/ICDT used the Zoom platform together with Slack. Pre-recorded talks of 10-12 minutes followed by live Q&A. It worked quite well IMO, but this is a much smaller conference.
Many talks were streamed live, that worked very well.
It is better to break one full day to two half a day, it is hard to sit in a chair wearing headphone for 8 hours.
having presentations uploaded and slides available quickly so you can view talk/tutorial within a short time if you cannot make the live event.
Hosting panels and doing Q&A (with moderator) was very smooth. Moderator would introduce, some talks in sequence with slides, you could pause the talk to take a note, then they could switch to a panel form (worked with up to 8? people) where the moderator could ask questions submitted (somehow, not sure) from the audience as well. A real-time voting mechanism on questions could be useful to help bubble questions to the top rather than having the moderator/staff have to look through tons of potential questions and select on the fly.
Tha major problem for me was timing the major parts of the conference were 4AM my time. My talk was prepared as a video and I have responded to questions on-line.
Scheduling made no allowance for different time zones, which was poor. The strict attendance limits made no sense for a virtual conference. It was hard (essentially impossible) to switch between sessions, or to attend when you had an hour available unexpectedly. Q&A was bad, because I couldn't read the other attendees questions, I had to wait till they were spoken, which is especially stupid.
Having a moderator during the talk (gathering the questions and asking them when they find appropriate).
Everything went well, except for questions by the audience.
Asking questions in the chat
In my opinion, the conference was a success. There were a few internet glitches but nothing that disrupted the flow of the conversations.
I like they provided a list of all papers, slides, and recorded videos. But I don't like they separated discussions (on Discord) from them (papers, slides, and videos).
I think the interactive nature of the conference, Q&A + chat integrated into streams provided a great experience, I mentioned this earlier but I liked the tool they used: Crowdcast.
it was organized very last minute (organizers were hoping to have a go at the conference before COVID onset on NYC), and the live speaker portion went reasonably well.
The good ones keep it short and concise with a clear focus. They are also narrow, with attendants I know from before and whom I've met socially.

Q22. Based on your past experience, what were the biggest pain points?

Timing of the conference that fits all participants and exact schedule.
The biggest benefit of conferences, which is enabling meeting other folks quite easily definitely didn't replicate. I don't have a solution for it, but it would be nice to hunt for creating solutions to this.
Q&A was terrible: stilted, slow, tedious. Networking was obviously non-existent.
Scheduling and protecting time to attend is difficult/impossible given other expectations
- Interpretation between languages. - Session time for my sleeping time.
Interaction with the audience
Questions by the audience. where typing was slower than asking. it would be perfect if the tool allows you to raise your hand (virtually) then ask your question verbally.
could not send questions to speaker via chat to session chair
- different timezones (it sounds stupid, but I am working in many timezones, and going back and forth while listening in to a conference is error prone) - having an agenda in a different tab and permanently going back and forth and guessing which talk is currently happening and how long it is going to last. - it would be great to have some timeline in the same window to be able to immediately see what is going on, how long it will be going on, when the next talk will be, what the different options are etc.
attention drops at some point
Loss of audio or poor audio of the speaker. Bad placement of videos sources if more than 1. For example, placing a feed of the speaker in the bottom corner somewhere on top of the slides, which nicely covers the main point of the slides. Don't overlap anything - do it e.g. like CPPcon https://www.youtube.com/channel/UCMlGfpWw-RUdWX_JbLCukXg
Some presenters' limited internet connectivity hindered Q&A
Not really being able to assess audience reaction.
Bad audio and lighting on the recording of the talk
- materials/slides were not shared in advance - the video feed of the slides was blurry
Zoom-bombing was a big issue
Limited seating in sessions
Not quite organized, and the access to other sessions.
Networking was almost impossible.
It seems coordinating time zones, and getting people to actively be a part of all parts of the conference (and not just show up for their talks) will be key.
No networking or socializing
Video quality, it differs widely between participants.
Interactivity with speakers, video lags and live streaming only modes
asking questions; it would have been better to use sli.do with the session chairperson picking up the questions there.
no pain from my side organizers put a too late deadline to deliver materials and were in a hurry close to the deadline
Presenters should remain online to answer questions related to his work
Their website was confusing from the getgo- unclear what the actual schedule is. Wasn't a clear way to separate live vs. pre-recorded. Just posting a poster pdf isn't that useful- if you're submitting a poster you should also include a 5min video.
IMO social/socializing side does not work at all. You just cannot replace conference lunches/dinners/receptions/hanging out in foyer/etc. Please be very careful not to create lame social events (team trivia, scavenger hunt or such). IMO it's very easy to make this look insulting and waste of time. Spontaneous (virtual) groups based on shared interests - e.g. gaming, (AI) philosophy, (AI) ethics, etc. - might be partial solution and might work somewhat. This kinda happened at ICML 2019 on Whova even though it was non-virtual conference.
Unable to hear the audience. Later learnt that the audience was muted and the onsite team had forgotten about this.
Workshops falling well outside my time windows.
No real socialization or networking
technical difficulties take too long, a speaker breaking up (ie audio and video), lack of TAs/ session support
When presenting, everybody being muted, and getting no feedback during the presentation feels like you are speaking to yourself, and it is impossible to estimate if the people are able to follow what you are telling.
Informal meetups, simulating 'corridor-interactions', the introduction of speaker/participant (too quick for something fickle like video streaming)
I can't see all the faces. QAs can be harder. timings. Hard to take leave for a virtual conference when you know things are going to be recorded. Not everything was recorded.
At the end of the talks, asking questions in private is painful in Zoom. Especially the speaker leaves early and does not have time to answer questions.
I would have liked to see more live talks and live face to face QA instead of relying only on Slack for QA.
Difficult to ask questions, rewind briefly if you miss what somebody said in a talk/panel, zero interaction with other participants or possibility to follow up with speakers etc. Wasn't the case for this conference, but it can be really annoying when a person does not bother to bring slides to help follow along (depending on the topic - if it is something like an interview or their personal reflections, different story). It is also annoying when you cannot consult each individual talk video after the fact (soon), and with separate slides if you need them - not providing PDFs of the slides, not hosted on a strange website that's hard to navigate, is frustrating. Just raw PDF is best.
Lack of in-room interactive experience. Chime sucks for lack of features like raising hand, no virtual backgrounds etc
Distributing questions fairly among the participants is a challenge. There might be many questions and MC need to decide which one to go with. I guess it is partially the case with the real scenarios as well but virtual ones are more challenging.
Even with all technology perfectly set up, it is very important to inform participants, get them involved and make sure the technology is actually used. There are no corridors where people can meet by accident or one can ask an organizer for a simple instruction.
its good for one to have a standby power supply especially when you are doing a presentation to make sure if power goes off, the presenter will not be interrupted
poster sessions are very challenging (were basically skipped).
There were up to 2500 people watching the conference at the same time, so the chat was crowded.
Things often change real time, but communication channels and validation of changes are often unclear. Hence working the communication channels beforehand will go a long way to smooth adaptation real time
Keeping the engagement with the live audience, which is not applicable here.
Internet disconnection
keep motivation, as ~3d intense 'real' conference becomes spread over 1w or more
There was no possible to ask questions for people who attended remotely. For the FAT* conference, only some of the talks were streamed live, other talks should have been uploaded later, but this didn't happen until today.
Really long schedule, hard to decide what to miss.
low attendance. some talks were only with the speakers from that time slot
Fatigue from being in front of a screen. Lack of access to recordings of talks.
Technical issues is by far the worst. On top of that virtual conferences are booring since they lack the natural social element.
Randomly bumping into people and having conversations, catching up with people I had met before in conferences.
The most important problem was that speakers sometimes would forget to switch on their mic and would not be audible to the audience during the entire talk (they wouldn't check messages in the chat either). This was problematic.
Online speakers being too fast or monotone, or their slides are overcrowded or not visible
Getting people's attention
As speaker I want to see the audience when I'm
One of the biggest pain points was that there were no opportunities to make new connections. There was a virtual drinking party at the conference, but I felt that it was difficult to talk to unknown people in a virtual environment (the party itself was interesting to me, though).
Time zone difference; not getting distracted
Q&A sessions were somewhat awkward.
The initial orientation teething problems. The organisers should have had a centralised information page for authors presenting. Emails with important information did not get to some workshop participants until late which cause a bit of panic. I think a page that only authors have access to with clear instructions/requirements listed would be very beneficial.
It's hard to recreate spontaneously meeting people who go to the same talks that I'm interested in.
Presenters with poor internet
someone was up at 4am to participate
Recorded talks taking a long time to be available
Connectivity during live streaming on the speaker side