The fast-growing machine learning as a service industry has incubated many APIs for multi-label classification tasks such as OCR and multi-object recognition. The heterogeneity in those APIs' price and performance, however, often forces users to choose between accuracy and expense. In this work, we propose FrugalMCT, a principled framework that jointly maximizes the accuracy while minimizes the expense by adaptively selecting the APIs to use for different data. FrugalMCT combines different APIs' predictions to improve accuracy and selects which combination to use to respect expense constraints. Preliminary experiments using ML APIs from Google, Microsoft, and other providers for multi-label image classification show that FrugalMCT often achieves more than 50% cost reduction while matching the accuracy of the best single API.

Authors: Lingjiao Chen ( Stanford University ) James Zou ( Stanford University ) Matei Zaharia ( Stanford and Databricks )