Timezone: »

 
Poster
Internet Explorer: Targeted Representation Learning on the Open Web
Alexander Li · Ellis Brown · Alexei Efros · Deepak Pathak

Tue Jul 25 02:00 PM -- 04:30 PM (PDT) @ Exhibit Hall 1 #610
in Poster Session 1 »
Event URL: https://internet-explorer-ssl.github.io/ »

Vision models typically rely on fine-tuning general-purpose models pre-trained on large, static datasets. These general-purpose models only capture the knowledge within their pre-training datasets, which are tiny, out-of-date snapshots of the Internet---where billions of images are uploaded each day. We suggest an alternate approach: rather than hoping our static datasets transfer to our desired tasks after large-scale pre-training, we propose dynamically utilizing the Internet to quickly train a small-scale model that does extremely well on a target dataset. Our approach, called Internet Explorer, explores the web in a self-supervised manner to progressively find relevant examples that improve performance on a desired target dataset. It cycles between searching for images on the Internet with text queries, self-supervised training on downloaded images, determining which images were useful, and prioritizing what to search for next. We evaluate Internet Explorer across several datasets and show that it outperforms or matches CLIP oracle performance using just a single GPU desktop to actively query the Internet for 30-40 hours.

[ PDF]  [ Poster] 

Author Information

Alexander Li (Carnegie Mellon University)
Ellis Brown (Carnegie Mellon University)
Ellis Brown

I am an incoming PhD student at NYU advised by Saining Xie and collaborating with Rob Fergus. I just finished a Master’s in Computer Science at CMU, where I was a graduate student researcher advised by Deepak Pathak and Alyosha Efros. Previously I was a founding engineer at BlackRock AI Labs, where I was fortunate to work with Mykel Kochenderfer, Stephen Boyd, and Trevor Hastie on projects in ML, optimization, and big data applied to finance. Before that, I studied computer science and mathematics at Vanderbilt.

Alexei Efros (UC Berkeley)
Deepak Pathak (Carnegie Mellon University)

More from the Same Authors