Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a single image of the desired physical product, which we apply to machine knitting. We propose to tackle this problem by directly learning to synthesize regular machine instructions from real images. We create a cured dataset of real samples with their instruction counterpart and propose to use synthetic images to augment it in a novel way. We theoretically motivate our data mixing framework and show empirical results suggesting that making real images look more synthetic is beneficial in our problem setup. We will make our dataset and code publicly available for reproducibility and to motivate further research related to manufacturing and program synthesis.
Alexandre Kaspar (MIT CSAIL)
PhD student at MIT CSAIL working on modeling tools for digital fabrication including 3D printing and industrial machine knitting.
Tae-Hyun Oh (MIT CSAIL)
Liane Makatura (MIT)
Petr Kellnhofer (MIT)
Wojciech Matusik (MIT)
Related Events (a corresponding poster, oral, or spotlight)
2019 Poster: Neural Inverse Knitting: From Images to Manufacturing Instructions »
Thu Jun 13th 01:30 -- 04:00 AM Room Pacific Ballroom
More from the Same Authors
2020 Poster: Efficient Continuous Pareto Exploration in Multi-Task Learning »
Pingchuan Ma · Tao Du · Wojciech Matusik
2020 Poster: Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control »
Jie Xu · Yunsheng Tian · Pingchuan Ma · Daniela Rus · Shinjiro Sueda · Wojciech Matusik