Expo Workshop

AI development today rests on three pillars: algorithms, hardware, and data. Ironically, the further AI moves towards new application areas, the more it depends on human efforts: more and more often data for training and validating AI models cannot be collected in any other way than by humans.

AI solutions require data for training and validating models that are not only high-quality and scalable to support growing industry needs but also flexible enough to support a large variety of use cases and data collection scenarios.

Toloka's mission is to create an environment for AI data production that is fully aligned with industry needs: quality, scalability, flexibility.

As a result, Toloka is a multifaceted solution with:

- a global crowdforce of 9 million Tolokers with around 200,000 active on the platform every month
- multiple methods and mechanisms for advanced automated quality control at scale, available for any platform using the Crowd-Kit library for Python
- instruments for integrating the crowd into the ML production process using the Toloka-Kit library for Python
- academic research and education initiatives in the field of Crowd Science for ML specialists

The Toloka workshop aims to cover these aspects and provide a comprehensive picture of how crowdsourcing can be applied to real life AI production.

Chat is not available.


Sun 5:00 a.m. - 5:30 a.m.
Evolution of data production paradigm in AI (Keynote)   
Olga Megorskaya
Sun 5:30 a.m. - 6:00 a.m.
The Practice of Crowdsourcing (Keynote)   
Omar Alonso
Sun 6:00 a.m. - 6:30 a.m.
Data Annotation at Scale: a Core Expertise of Modern ML (Keynote)   
Daria Baidakova
Sun 6:30 a.m. - 7:00 a.m.
The Future of Work for Performers: Empowering the People behind AI (Keynote)   
Saiph Savage
Sun 7:00 a.m. - 7:12 a.m.
Introduction, Problem Statement, and Task Design (Demonstration)   
Oleg Pavlov
Sun 7:12 a.m. - 7:24 a.m.
Programmatical Definition of Post-Acceptance Tasks with Toloka-Kit (Demonstration)   
Vladimir Losev
Sun 7:24 a.m. - 7:36 a.m.
Aggregating Categorical Replies with Crowd-Kit (Demonstration)   
Dmitry Ustalov
Sun 7:36 a.m. - 7:48 a.m.
Programmatical Definition of Side-by-Side Tasks with Toloka-Kit (Demonstration)   
Vladimir Losev
Sun 7:48 a.m. - 8:00 a.m.
Aggregating Pairwise Comparisons with Crowd-Kit (Demonstration)   
Dmitry Ustalov
Sun 8:00 a.m. - 9:00 a.m.
Live Q&A Session (Discussion Panel)