Agent Transparency and Human Performance in Supervisory Control

Abstract

Background: In future maritime transport, Artificial Intelligence (AI)-enabled systems may allow ships to sail without direct human involvement. Here, humans are foreseen to take a supervisory role to ensure performance and safety requirements are met. However, there are well-known challenges related to this role that may affect the operator’s ability to intervene. Agent transparency is a design principle aimed at supporting supervisory control by providing humans with insight into the system’s decisions, planned actions, and internal reasoning. However, considering the novelty of the application of AI-enabled systems in safety-critical domains, there is limited experience with the effect of transparency on human supervisory performance in these settings. Aim: The research aim is to explore the application of transparency to support supervisory control. Five research questions are investigated: 1. What is the relationship between agent transparency and Situation Awareness, mental workload, and task performance? 2. How is human performance achieved in conventional- and supervised maritime collision avoidance? 3. How does a model for human information processing form the basis for agent transparency in the ship autonomy context? 4. How should a maritime collision avoidance system be made transparent to a human supervisor? 5. What is the relationship between agent transparency and Situation Awareness, mental workload, and task performance in maritime autonomous collision avoidance manoeuvring? Method: The study applied a mix of quantitative and qualitative methods: 1. Systematic literature review (review of 17 peer-reviewed articles) 2. Goal-Directed Task Analysis (in situ observations & interviews, 11 navigators, COLREGs) 3. Modelling (adapted and contextualized a model for human information processing) 4. Human Machine Interface development (iterative design process, workshops, 5 navigators) 5. Controlled experiment (measuring human performance effects, 34 navigators) Results: The thesis outlined the following contributions: 1. Synthesis of the research state-of-the-art on agent transparency and human performance 2. Explored cognitive tasks, identified information requirements to support supervisory control 3. Established transparency model, created layers of information indicating system reasoning 4. Developed realistic traffic situations, applied transparency model to design interface concepts 5. Evaluated the model in the autonomous collision avoidance context, found effects on SA and task performance, but not on mental workload Conclusions: This dissertation advocates the relevance of affording human operators with insight into the reasoning of autonomous systems and establishes transparency as an important prerequisite on the path towards safe and effective human-supervisory control. With these new insights, meaningful human work may be created where the combined capabilities of human-agent teams can be optimised