AI development today rests on three pillars: algorithms, hardware, and data. Ironically, the further AI moves towards new application areas, the more it depends on human efforts: more and more often data for training and validating AI models cannot be collected in any other way than by humans.
AI solutions require data for training and validating models that are not only high-quality and scalable to support growing industry needs but also flexible enough to support a large variety of use cases and data collection scenarios.
Toloka's mission is to create an environment for AI data production that is fully aligned with industry needs: quality, scalability, flexibility.
As a result, Toloka is a multifaceted solution with:
- a global crowdforce of 9 million Tolokers with around 200,000 active on the platform every month
- multiple methods and mechanisms for advanced automated quality control at scale, available for any platform using the Crowd-Kit library for Python
- instruments for integrating the crowd into the ML production process using the Toloka-Kit library for Python
- academic research and education initiatives in the field of Crowd Science for ML specialists
The Toloka workshop aims to cover these aspects and provide a comprehensive picture of how crowdsourcing can be applied to real life AI production.
Sun 5:00 a.m. - 5:30 a.m.
|
Evolution of data production paradigm in AI
(
Keynote
)
>
SlidesLive Video |
Olga Megorskaya 🔗 |
Sun 5:30 a.m. - 6:00 a.m.
|
The Practice of Crowdsourcing
(
Keynote
)
>
SlidesLive Video |
Omar Alonso 🔗 |
Sun 6:00 a.m. - 6:30 a.m.
|
Data Annotation at Scale: a Core Expertise of Modern ML
(
Keynote
)
>
SlidesLive Video |
Daria Baidakova 🔗 |
Sun 6:30 a.m. - 7:00 a.m.
|
The Future of Work for Performers: Empowering the People behind AI
(
Keynote
)
>
SlidesLive Video |
Saiph Savage 🔗 |
Sun 7:00 a.m. - 7:12 a.m.
|
Introduction, Problem Statement, and Task Design
(
Demonstration
)
>
SlidesLive Video |
Oleg Pavlov 🔗 |
Sun 7:12 a.m. - 7:24 a.m.
|
Programmatical Definition of Post-Acceptance Tasks with Toloka-Kit
(
Demonstration
)
>
SlidesLive Video |
Vladimir Losev 🔗 |
Sun 7:24 a.m. - 7:36 a.m.
|
Aggregating Categorical Replies with Crowd-Kit
(
Demonstration
)
>
SlidesLive Video |
Dmitry Ustalov 🔗 |
Sun 7:36 a.m. - 7:48 a.m.
|
Programmatical Definition of Side-by-Side Tasks with Toloka-Kit
(
Demonstration
)
>
SlidesLive Video |
Vladimir Losev 🔗 |
Sun 7:48 a.m. - 8:00 a.m.
|
Aggregating Pairwise Comparisons with Crowd-Kit
(
Demonstration
)
>
SlidesLive Video |
Dmitry Ustalov 🔗 |
Sun 8:00 a.m. - 9:00 a.m.
|
Live Q&A Session
(
Discussion Panel
)
>
SlidesLive Video |
🔗 |