Human in the Loop Machine Learning

For details see:
http://machlearn.gitlab.io/hitl2017/

As machine learning systems become more ubiquitous in everybody’s day-to-day life or work, society and industry is in an intermediate state between fully manual and fully automatic systems. The gradient undoubtedly points towards full automation, but moving forward in this direction is going to face increasing challenges due to the fact that current machine learning research tends to focus on end to end systems, which puts aside the fact that for practical applications there are still gaps or caveats in the automation. Parts of these come from the presence of (or the necessity to have) the Human in the Loop.

There are two main locations for the Human in the automated system: (i) upstream, in which case the focus is mainly in the inputs of the algorithm. This can be essential for personalised assistants, that describe environments where the machine learning method is tightly embedded into the system. Such environments pose additional challenges related to privacy at large; (ii) downstream: other domains have machine learning approaches analyse parts of the data, and human experts use the results and intuition to make decisions.

The Human dependences between these two locations is also neither straightforward nor acyclic — some applications tend to have feedback effects on data as actions or interventions are undertaken based on machine learning predictions. Furthermore there are often very few rounds of decision making in practice, but each round may affect the statement of the problems related to the Human presence, as witnessed for example by eventual privacy leakages.

This workshop aims to bring together people who are working on systems where machine learning is only part of the solution. Participants will exchange ideas and experiences on human in the loop machine learning.

Topics of interest include:
- System architectures that allow for human decision making
- User interfaces for interacting with machine learning systems
- Validation of human in the loop software systems
- Viewpoints from traditional fields such as reinforcement learning and Bayesian optimisation
- Challenges related to the human presence in the loop (privacy, bias, fairness, etc.)
- Case studies of deployed machine learning