THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Talk
World of Bits: An Open-Domain Platform for Web-Based Agents
Tim Shi · Andrej Karpathy · Jim Fan · Jonathan Hernandez · Percy Liang

Tue Aug 08 10:48 PM -- 11:06 PM (PDT) @ Darling Harbour Theatre
in Applications »

While simulated game environments have greatly accelerated research in reinforcement learning, existing environments lack the open-domain realism of tasks in computer vision or natural language processing, which operate on artifacts created by humans in natural, organic settings. To foster reinforcement learning research in such settings, we introduce the World of Bits (WoB), a platform in which agents complete tasks on the Internet by performing low-level keyboard and mouse actions. The two main challenges are: (i) to curate a large, diverse set of interesting web-based tasks, and (ii) to ensure that these tasks have a well-defined reward structure and are reproducible despite the transience of the web. To do this, we develop a methodology in which crowdworkers create tasks defined by natural language questions and provide demonstrations of how to answer the question on real websites using keyboard and mouse; HTTP traffic is cached to create a reproducible offline approximation of the web site. Finally, we show that agents trained via behavioral cloning and reinforcement learning can successfully complete a range of our web-based tasks.

[ PDF]  [ ss Summary/Notes]  [ Video] 

Author Information

Tim Shi (Stanford University)
Andrej Karpathy (OpenAI)
Jim Fan (Stanford University)
Jonathan Hernandez
Percy Liang (Stanford University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors