Adapting Blackbox Generative Models via Inversion

Adapting large-scale generative AI tools to differ-ent end uses continues to be challenging, as manyindustry grade image generator models are notpublicly available. Thus, to finetune an industrygrade image generator is not currently feasiblein the classical sense of finetuning certain layersof a given deep-network. Instead, we present analternative perspective for the problem of adapt-ing large-scale generative models that does notrequire access to the full model. Recognizingthe expense of storing and fine-tuning generativemodels, as well as the restricted access to weightsand gradients (often limited to API calls only), weintroduce AdvIN (Adapting via Inversion). Thisapproach advocates the use of inversion methods,followed by training a latent generative model asbeing equivalent to adaptation. We evaluate thefeasibility of such a framework on StyleGANswith real distribution shifts, and outline someopen research questions. Even with simple in-version and latent generation strategies, AdvINis surprisingly competitive to fine-tuning basedmethods, making it a promising alternative forend-to-end fine-tuning