Workshop on Human-Machine Collaboration and Teaming

Umang Bhatt · Katie Collins · Maria De-Arteaga · Bradley Love · Adrian Weller

Ballroom 4
Abstract Workshop Website
Sat 23 Jul, 5:55 a.m. PDT

Abstract:

Machine learning (ML) approaches can support decision-making in key societal settings including healthcare and criminal justice, empower creative discovery in mathematics and the arts, and guide educational interventions. However, deploying such human-machine teams in practice raises critical questions, such as how a learning algorithm may know when to defer to a human teammate and broader systemic questions of when and which tasks to dynamically allocate to a human versus a machine, based on complementary strengths while avoiding dangerous automation bias. Effective synergistic teaming necessitates a prudent eye towards explainability and offers exciting potential for personalisation in interaction with human teammates while considering real-world distribution shifts. In light of these opportunities, our workshop offers a forum to focus and inspire core algorithmic developments from the ICML community towards efficacious human-machine teaming, and an open environment to advance critical discussions around the issues raised by human-AI collaboration in practice.

Chat is not available.
Timezone: America/Los_Angeles »

Schedule

 Sat 5:55 a.m. - 6:00 a.m. Welcome and Introduction (Introduction) 🔗 Sat 6:00 a.m. - 6:25 a.m. Machine-only to human-machine collaboration from practical AI deployments. Ernest Mwebaze (Invited Talk)    A strong temptation in the "AI for social good" space is a bias towards more efficient solutions through automation or a machine-only intervention. In this talk I give several examples where going in with this assumption results in sub-optimal solutions and the need for a human-machine collaboration becomes evident. A side effect of this is usually a move from simple automation to a more context-aware design of a potential solution. I highlight the need for considering context as an integral factor in human-machine collaboration through examples from practical deployments of AI solutions in the developing world context. 🔗 Sat 6:25 a.m. - 6:30 a.m. Q&A for Ernest (Q&A) 🔗 Sat 6:30 a.m. - 7:00 a.m. Spotlight Paper Flashtalks (Recorded Flash Talks) 🔗 Sat 7:00 a.m. - 7:30 a.m. Discussion. Deploying Human-Machine Teams in Practice (Ernest and Wendy) (Discussion Panel) 🔗 Sat 7:30 a.m. - 7:45 a.m. Coffee Break and Chat (Break) 🔗 Sat 7:45 a.m. - 8:10 a.m. Inside and Outside: Ways to Control AI Systems. Fernanda Viegas and Martin Wattenberg (Invited Talk) 🔗 Sat 8:10 a.m. - 8:15 a.m. Q&A for Fernanda and Martin (Q&A) 🔗 Sat 8:15 a.m. - 8:40 a.m. Creating Human-Computer Partnerships. Wendy Mackay (Invited Talk)    One of the key differences between AI and HCI research is that AI measures success in terms of more effective algorithms, whereas HCI focuses on improving interaction and enhancing human skills over time. I argue that better AI algorithms are neither necessary nor sufficient for creating more effective intelligent systems. Instead, we need to create human-computer partnerships that take advantage of machine learning but leave the user in control. I describe several projects that use generative theories of interaction to design intelligent interactive systems that users find discoverable, expressive and appropriable. 🔗 Sat 8:40 a.m. - 8:45 a.m. Q&A for Wendy (Q&A) 🔗 Sat 8:45 a.m. - 9:10 a.m. Towards Human-Centric Human-Machine Interaction. Nuria Oliver (Invited Talk)    In my talk, I will present two ideas related to human-machine collaboration which emphasize the human-element. The objective is to develop human+AI systems that support and enhance the human experience. In particular, I will advocate for the development of intelligent systems that (1) help us embrace boredom and (2) understand, model and possibly mimic human cognitive biases We have implemented the first idea whereas the second idea is a new research area that we started just a couple of months ago. 🔗 Sat 9:10 a.m. - 9:15 a.m. Q&A for Nuria (Q&A) 🔗 Sat 9:15 a.m. - 10:05 a.m. Lunch Break (Break) 🔗 Sat 10:05 a.m. - 10:45 a.m. How Will Interactive Theorem Provers Develop? Sir Timothy Gowers (Recorded Talk, but with Live Q&A at 13:30!) (Invited Talk) 🔗 Sat 10:40 a.m. - 11:40 a.m. Poster Session 🔗 Sat 11:40 a.m. - 11:50 a.m. Human-AI Collaboration in Decision-Making: Beyond Learning to Defer. Diogo Leitao (Contributed Live Talks)    Human-AI collaboration (HAIC) in decision-making aims to improve predictive performance, fairness and efficiency by creating synergistic teaming between human decision-makers and AI systems. A key challenge in HAIC is to determine who among AI and humans takes which decisions. Recently, learning to defer (L2D) has been presented as a promising approach to tackle this challenge. Nevertheless, L2D entails several often unfeasible requirements, such as the availability of predictions from human decision-makers for every instance or ground-truth labels independent from said decision-makers. Furthermore, neither L2D nor alternative approaches tackle fundamental issues of deploying HAIC into real-world scenarios, such as capacity management or non-stationarity. In this paper, we aim to identify these and other limitations, pointing to where opportunities for future research work in HAIC may lie. 🔗 Sat 11:50 a.m. - 12:00 p.m. Argumentative reward learning: Reasoning about human preferences. Francis Rhys Ward (Contributed Recorded Talk)    We define a novel neuro-symbolic framework, argumentative reward learning, which combines preference-based argumentation with existing approaches to reinforcement learning from human feedback. Our method improves prior work by generalising human preferences, reducing the burden on the user and increasing the robustness of the reward model. We demonstrate this with a number of experiments. 🔗 Sat 12:00 p.m. - 12:45 p.m. Coffee Break and Chat (Break) 🔗 Sat 12:45 p.m. - 1:30 p.m. Panel/Discussion. Human-Machine Teams for Mathematicians (Igor, Tony, Talia, and Petar) (Discussion Panel) 🔗 Sat 1:30 p.m. - 1:40 p.m. Machine Explanations and Human Understanding. Chacha Chen (Contributed Live Talks)    Explanations are hypothesized to improve human understanding of machine learning models. However, empirical studies have found mixed and even negative results. It remains an open question what factors drive such mixed results. To address this question, we first conduct a literature survey and identify three core concepts that cover all existing quantitative measures of understanding: task decision boundary, model decision boundary, and model error. We argue that human intuitions are necessary for generating and evaluating explanations in human-AI decision making: without assumptions about human intuitions, explanations may potentially improve human understanding of model decision boundary, but they cannot improve human understanding of task decision boundary or model error (see formal discussions in the appendix). We further validate the importance of human intuitions in shaping the outcome of machine explanations with empirical human subject studies. 🔗 Sat 1:40 p.m. - 1:50 p.m. Human-machine collaboration for reusable and scalable models of remote sensing imagery analysis. Lexie Yang (Contributed Live Talks)    In this paper, we describe efforts towards a continual learning workflow for automated building segmentation from remote sensing imagery that is intended to support decision making by governmental agencies. We describe a workflow under development that leverages human-machine collaboration as a core strategy to decrease annotation costs/needs, as well as aid quality assurance and increase understandability from end-users without necessarily machine learning expertise. These aspects will potentially increase trust by intended users and overall impact of model outcomes on real-world decision making. 🔗 Sat 1:50 p.m. - 2:00 p.m. How to Talk so Robots will Learn: Instructions, Descriptions, Alignment. Ted Sumers (Contributed Live Talks)    From the earliest years of our lives, humans use language to express our beliefs and desires. Being able to talk to artificial agents about our preferences would thus fulfill a central goal of value alignment. Yet today, we lack computational models explaining such flexible and abstract language use. To address this challenge, we consider social learning in a linear bandit setting and ask how a human might communicate preferences over behaviors (i.e. the reward function). We study two distinct types of language: instructions, which ground to concrete actions, and descriptions, which provide information about the reward function. To explain how humans use these forms of language, we suggest they reason about both known present and unknown future states: instructions optimize for the present, while descriptions optimize for the future. We formalize this choice by extending reward design to consider a distribution over states. We then define a pragmatic listener agent that infers the speaker's reward function by reasoning about how the speaker expresses themselves. Our findings suggest that (1) descriptions afford stronger generalization than instructions; and (2) the notion of a latent speaker horizon allows for more robust value alignment from natural language input. We hope these insights can help broaden the field's focus on instructions to study more abstract, descriptive language. 🔗 Sat 2:00 p.m. - 2:00 p.m. Closing Statements (Closing) 🔗 - Perspectives on Incorporating Expert Feedback into Model Updates (Poster) Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with open questions that naturally arise from our proposed taxonomy and subsequent survey. Umang Bhatt 🔗 - Machine Explanations and Human Understanding (Poster)    Explanations are hypothesized to improve human understanding of machine learning models. However, empirical studies have found mixed and even negative results. It remains an open question what factors drive such mixed results. To address this question, we first conduct a literature survey and identify three core concepts that cover all existing quantitative measures of understanding: task decision boundary, model decision boundary, and model error. We argue that human intuitions are necessary for generating and evaluating explanations in human-AI decision making: without assumptions about human intuitions, explanations may potentially improve human understanding of model decision boundary, but they cannot improve human understanding of task decision boundary or model error (see formal discussions in the appendix). We further validate the importance of human intuitions in shaping the outcome of machine explanations with empirical human subject studies. Chacha Chen 🔗 - Argumentative reward learning: Reasoning about human preferences (Poster) We define a novel neuro-symbolic framework, argumentative reward learning, which combines preference-based argumentation with existing approaches to reinforcement learning from human feedback. Our method improves prior work by generalising human preferences, reducing the burden on the user and increasing the robustness of the reward model. We demonstrate this with a number of experiments. Francis Rhys Ward 🔗 - Human-AI Collaborative Decision-Making: Beyond Learning to Defer (Poster)    Human-AI collaboration (HAIC) in decision-making aims to improve predictive performance, fairness and efficiency by creating synergistic teaming between human decision-makers and AI systems. A key challenge in HAIC is to determine who among AI and humans takes which decisions. Recently, learning to defer (L2D) has been presented as a promising approach to tackle this challenge. Nevertheless, L2D entails several often unfeasible requirements, such as the availability of predictions from human decision-makers for every instance or ground-truth labels independent from said decision-makers. Furthermore, neither L2D nor alternative approaches tackle fundamental issues of deploying HAIC into real-world scenarios, such as capacity management or non-stationarity. In this paper, we aim to identify these and other limitations, pointing to where opportunities for future research work in HAIC may lie. Diogo Leitao · Pedro Saleiro 🔗 - Human-machine collaboration for reusable and scalable models for remote sensing imagery analysis (Poster)    In this paper, we describe efforts towards a continual learning workflow for automated building segmentation from remote sensing imagery that is intended to support decision making by governmental agencies. We describe a workflow under development that leverages human-machine collaboration as a core strategy to decrease annotation costs/needs, as well as aid quality assurance and increase understandability from end-users without necessarily machine learning expertise. These aspects will potentially increase trust by intended users and overall impact of model outcomes on real-world decision making. Philipe Ambrozio Dias 🔗 - Counterfactual Inference of Second Opinions (Poster)    Automated decision support systems that are able to infer second opinions from experts can potentially facilitate a more efficient allocation of resources—they can help decide when and from whom to seek a second opinion. In this paper, we look at the design of this type of support systems from the perspective of counterfactual inference. We focus on a multiclass classification setting and first show that, if experts make predictions on their own, the underlying causal mechanism generating their predictions needs to satisfy a desirable set invariant property. Further, we show that, for any causal mechanism satisfying this property, there exists an equivalent mechanism where the predictions by each expert are generated by independent sub-mechanisms governed by a common noise. This motivates the design of a set invariant Gumbel-Max structural causal model where the structure of the noise governing the sub-mechanisms underpinning the model depends on an intuitive notion of similarity between experts which can be estimated from data. Experiments on both synthetic and real data show that our model can be used to infer second opinions more accurately than its non-causal counterpart. Nina Corvelo Benz 🔗 - A Human-Centric Assessment Framework for AI (Poster)    With the rise of AI systems in real-world applications comes the need for reliable and trustworthy AI. An important aspect for this are explainable AI systems. However, there is no agreed standard on how explainable AI systems should be assessed. Inspired by the Turing test, we introduce a human-centric assessment framework where a leading domain expert accepts or rejects the solutions of an AI system and another domain expert. By comparing the acceptance rates of provided solutions, we can assess how the AI system performs in comparison to the domain expert, and in turn whether or not the AI system's explanations (if provided) are human understandable. This setup---comparable to the Turing test---can serve as framework for a wide range of human-centric AI system assessments. We demonstrate this by presenting two instantiations: (1) an assessment that measures the classification accuracy of a system with the option to incorporate label uncertainties; (2) an assessment where the usefulness of provided explanations is determined in a human-centric manner. Sascha Saralajew 🔗 - Predicting Human Similarity Judgments Using Large Language Models (Poster) Similarity judgments provide a well-established method for accessing mental representations, with applications in psychology, neuroscience and machine learning. However, collecting similarity judgments can be prohibitively expensive for naturalistic datasets as the number of comparisons grows quadratically in the number of stimuli. We leverage recent advances in language models and online recruitment, proposing an efficient domain-general procedure for predicting human similarity judgments based on text descriptions. Crucially, the number of descriptions required grows only linearly with the number of stimuli, drastically reducing the amount of data required. We test this procedure on six datasets of naturalistic images and show that our models outperform previous approaches based on visual information. Raja Marjieh 🔗 - CrowdPlay: Crowdsourcing demonstrations for learning human-AI interaction (Poster)    Crowdsourcing has been instrumental for driving AI advances that rely on large-scale data. At the same time, reinforcement learning has seen rapid progress through the development of an almost plug-and-play software ecosystem around standard libraries such as OpenAI Gym and Baselines. In this paper, we aim to fill a gap at the intersection of these two: Enabling large-scale collection of human behavioral data in standard AI environments and together with AI agents trained with standard libraries, with the aim of enabling novel advancements in offline learing and human-AI interaction research. To this end, we present CrowdPlay, a complete crowdsourcing pipeline for any standard RL environment including OpenAI Gym (made available under an open-source license) and a large-scale publicly available crowdsourced dataset of human gameplay demonstrations in Atari 2600 games, including human-AI multiagent data. For pairing human and AI agents in the same environment, CrowdPlay can directly interface with standard RL training pipelines, allowing deployment of trained agents with minimal overhead. We hope that this will drive the improvement in design of algorithms that account for the complexity of human, behavioral data, and as a platform for evaluation of human-AI cooperation methods. Our code and dataset are available under at (URL redacted for blind review). Matthias Gerstgrasser 🔗 - Elicit: A Framework for Human-in-the-Loop High-Precision Information Extraction from Text Documents (Poster)    Extracting information from unstructured text can help build new datasets and facilitate valuable research. Weak supervision methods can produce impressive results but may not be sufficiently reliable for high-stakes applications where precision is essential. We present a framework for information extraction which adds a human-in-the-loop element to weak supervision labelling. We demonstrate our approach by creating two new datasets with information on criminal trials from publicly available legal documents and news articles. We show that our approach requires much less human effort than manual information extraction while achieving comparable precision. Bradley Butcher 🔗 - Training Novices: The Role of Human-AI Collaboration and Knowledge Transfer (Poster)    In various work environments expert knowledge is pivotal for humans to conduct tasks with high performance and ensure business success. These humans possess task-specific expert knowledge (TSEK) and hence, represent subject matter experts (SMEs). However, not only demographic changes but also personnel downsizing strategies lead and will continue to lead to departures of SMEs within organizations, which induces challenges on how to retain that expert knowledge and train novices to keep the competitive advantage elicited by that expert knowledge. The training of novices by SMEs is time and cost intensive and intensifying the need for alternatives. Human-AI collaboration (HAIC) poses a way out of this dilemma, facilitating alternatives to preserve expert knowledge and teach it to novices for tasks conducted by SMEs beforehand. In this workshop paper, we (1) propose a framework on how HAIC can train novices particular tasks, (2) illustrate the role of explicit and tacit knowledge in this training process via HAIC, and (3) illustrate a preliminary experiment design to assess the ability of AI systems in HAIC to act as a teacher to transfer TSEK to novices who do not possess prior TSEK. Philipp Spitzer 🔗 - Towards Effective Case-Based Decision Support with Human-Compatible Representations (Poster)    Algorithmic case-based decision support provides examples to help humans make sense of predicted labels and aid humans in decision-making tasks. However, representations learned by supervised models may not align well with human intuitions: what models consider as similar examples can be perceived as distinct by humans. In this work, we incorporate ideas from metric learning with supervised learning to examine the importance of alignment for effective decision support. In addition to instance-level labels, we use human-provided triplet judgments to learn human-compatible decision-focused representation. Using human subject experiments, we demonstrate that such representation is better aligned with human perception than representation solely optimized for classification. Human-compatible representations identify nearest neighbors that are perceived as more similar by humans and allow humans to make more accurate predictions. Han Liu 🔗 - Adaptive Out-of-Distribution Detection with Human-in-the-Loop (Poster)    Robustness to Out-of-Distribution (OOD) samples is essential for successful deployment of machine learning models in the open world. Many existing approaches focus on offline setting and maintaining a true positive rate (TPR) of 95% which is usually achieved by using an uncertainty score with a threshold based on ID data available for training the models. In contrast, practical systems have to deal with OOD samples on the fly (online setting) and many critical applications, e.g., medical diagnosis, demand the system to meet quality constraints in terms of controlling FPR (false positive rate) at most 5%. This is challenging since having adequate access to the variety of OOD data, the system encounters after deployment is hard. To meet this challenge, we propose a human-in-the-loop system for OOD detection that can adapt to variations in the OOD data while adhering to the quality constraints. Our system is based on active learning approaches and is complementary to the current OOD-detection methods. We evaluate our system empirically on a mixture of benchmark OOD datasets in image classification task on CIFAR-10 and show that our method can maintain FPR at most 5% while maximizing TPR and making a limited number of human queries. Heguang Lin 🔗 - Learning to Play with the Machines in Social Science Research: Bringing the Theory Back In (Poster)    In this position paper, we argue that computational social science, as an evolving field, lacks and requires organizing principles. A broad space separates the two constituent disciplines, which has to date been sidestepped rather than filled by applying increasingly complex computational models to problems in social science research. We review the three major questions about the disciplines' cooperation. We thereby identify that the challenges are not practical but epistemological. The crisis of a lack of theory began at the field's inception and has, over the decades, grown more important and challenging. A return to theoretically grounded research questions will propel the field from both theoretical and methodological points of view. Giovanna Maria Dora Dore 🔗 - A Framework for Learning to Request Rich and Contextually Useful Information from Humans (Poster)    Leveraging human assistance can help agents overcome their inherent limitations and robustly cope with unfamiliar situations. We present a general interactive framework that enables an agent to determine and request contextually useful information from an assistant, and to incorporate rich forms of responses into its decision-making process. We demonstrate the practicality of our framework on a simulated human-assisted navigation problem. Aided with an assistance-requesting policy learned by our method, a navigation agent achieves up to a 7× improvement in success rate on tasks that take place in previously unseen environments, compared to fully autonomous behavior. Khanh Nguyen 🔗 - Effects of Algorithmic Fairness Constraints on Human Hiring Decisions (Poster) Despite the explosion of scholarship in algorithmic fairness, little is understood about how algorithmic fairness constraints interact with human decisions. In this paper, we present and solve a 2-stage hiring model to understand the interplay between algorithmic fairness constraints and human hiring decisions. We consider a hiring scenario in which a diversity-conscious company seeks to hire an employee from a set of applicants. There are more male than female applicants, but both have the same underlying quality distribution. In the first stage, a screening algorithm screens and shortlists candidates. To improve the gender diversity of the workforce, the algorithm has a gender-parity constraint such that it shortlists an equal number of men and women. In the second stage, an unbiased hiring manager interviews the shortlisted candidates and hires the best candidate based on her assessment. We solve this model analytically and identify 3 key parameters that affect the gender proportion of the hires: (1) the size of the applicant pool, (2) the correlation between the algorithm's and the hiring manager's assessment, and (3) the difference in the screening algorithm's predictive power between female and male candidates. Prasanna Parasurama 🔗 - Diverse Concept Proposals for Concept Bottleneck Models (Poster)    Concept bottleneck models are interpretable predictive models that are often used in domains where model trust is a key priority, such as healthcare. They identify a small number of human-interpretable concepts in the data, which they then use to make predictions. Learning relevant concepts from data proves to be a challenging task. The most predictive concepts may not align with expert intuition, thus, failing interpretability with no recourse. Our proposed approach identifies a number of predictive concepts that explain the data. By offering multiple alternative explanations, we allow the human expert to choose the one that best aligns with their expectation. To demonstrate our method, we show that it is able discover all possible concept representations on a synthetic dataset. On EHR data, our model was able to identify 4 out of the 5 pre-defined concepts without supervision. Katrina Brown 🔗 - A Human-Centric Take on Model Monitoring (Poster)    Predictive models are increasingly used to make various consequential decisions in high-stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that these models make accurate predictions, are robust to shifts in the data, do not rely on spurious features, and do not unduly discriminate against minority groups. To this end, several approaches spanning various areas such as explainability, fairness, and robustness have been proposed in recent literature. Such approaches need to be human-centered as they cater to the understanding of the models to their users. However, there is a research gap in understanding the human-centric needs and challenges of monitoring machine learning (ML) models once they are deployed. To fill this gap, we conducted an interview study with 13 practitioners who have experience at the intersection of deploying ML models and engaging with customers spanning domains such as financial services, healthcare, hiring, online retail, computational advertising, and conversational assistants. We identified various human-centric challenges and requirements for model monitoring in real-world applications. Specifically, we found the need and the challenge for the model monitoring systems to clarify the impact of the monitoring observations on outcomes. Further, such insights must be actionable, robust, customizable for domain-specific use cases, and cognitively considerate to avoid information overload. Murtuza Shergadwala 🔗 - On the Calibration of Learning to Defer to Multiple Experts (Poster)    We study the calibration properties of multi-expert learning to defer (L2D). In particular, we study the framework's ability to estimate $\mathbb{P}(\rsm_{j} = \ry | \vx)$, the probability that the $j$th expert will correctly predict the label for $\vx$. We compare softmax- and one-vs-all-parameterized L2D, finding the former causes mis-calibration to propagate between the estimates of expert correctness while the latter's parameterization does not. Rajeev Verma 🔗 - The Influence of Explainable Artificial Intelligence: Nudging Behaviour or Boosting Capability? (Poster)    This article aims to provide a theoretical account and corresponding paradigm for analysing how explainable artificial intelligence (XAI) influences people's behaviour and cognition. It uses insights from research on behaviour change. Two notable frameworks for thinking about behaviour change techniques are nudges - aimed at influencing behaviour - and boosts - aimed at fostering capability. It proposes that local and concept-based explanations are more adjacent to nudges, while global and counterfactual explanations are more adjacent to boosts. It outlines a method for measuring XAI influence and argues for the benefits of understanding it for optimal, safe and ethical human-AI collaboration. Matija Franklin 🔗 - Bayesian Weak Supervision via an Optimal Transport Approach (Poster)    Large-scale machine learning is often impeded by a lack of labeled training data. To address this problem, the paradigm of weak supervision aims to collect and then aggregate multiple noisy labels. We propose a Bayesian probabilistic model that employs optimal transport to derive a ground-truth label. The translation between true and weak labels is cast as a transport problem with an inferred cost structure. Our approach achieves strong performance on the WRENCH weak supervision benchmark. Moreover, the posterior distribution over cost matrices allows for exploratory analysis of the weak sources. Putra Manggala 🔗 - A Taxonomy Characterizing Human and ML Predictive Decision-making (Poster)    Hybrid human-ML systems are increasingly in charge of consequential decisions in a wide range of domains. Drawing upon prior literature in human psychology, machine learning, and human-computer interaction, we introduce a taxonomy characterizing a wide variety of criteria across which human and machine decision-making differ. Our taxonomy offers a critical piece of insight for future work in human-ML collaborative decision-making:the mechanism by which we combine human-ML judgments should be informed by the underlying causes of their diverging decisions. Charvi Rastogi 🔗 - How to Talk so Robots will Learn: Instructions, Descriptions, and Alignment (Poster) From the earliest years of our lives, humans use language to express our beliefs and desires. Being able to talk to artificial agents about our preferences would thus fulfill a central goal of value alignment. Yet today, we lack computational models explaining such flexible and abstract language use. To address this challenge, we consider social learning in a linear bandit setting and ask how a human might communicate preferences over behaviors (i.e. the reward function). We study two distinct types of language: instructions, which ground to concrete actions, and descriptions, which provide information about the reward function. To explain how humans use these forms of language, we suggest they reason about both known present and unknown future states: instructions optimize for the present, while descriptions optimize for the future. We formalize this choice by extending reward design to consider a distribution over states. We then define a pragmatic listener agent that infers the speaker's reward function by reasoning about how the speaker expresses themselves. Our findings suggest that (1) descriptions afford stronger generalization than instructions; and (2) the notion of a latent speaker horizon allows for more robust value alignment from natural language input. We hope these insights can help broaden the field's focus on instructions to study more abstract, descriptive language. Theodore R Sumers 🔗