Proactive self-adaptation and assurance of explainable Human–Machine Teaming

With the pervasiveness of software-intensive systems, autonomous agents are increasingly involved in missions entailing close cooperation with humans in critical domains. Human–Machine Teaming (HMT) requires agents to have a high degree of autonomy to make dependable decisions in safety-critical settings and quickly react to unexpected changes in the environment. We present AdaptEASE, a framework that leverages formal modeling and verification to model HMT missions offline and predict the mission outcome at runtime through Machine Learning (ML) models trained on Statistical Model Checking (SMC) results. The framework also exploits explainable AI techniques to inform stakeholders on what factors impact the predicted outcome positively or negatively. Building upon previous work, AdaptEASE enhances the controlled autonomous agents with the capability to self-adapt at runtime proactively. Specifically, if an unfavorable situation emerges during the HMT, AdaptEASE identifies an adaptation action that guarantees balance between the agents’ performance and the involved human subjects’ comfort levels. While runtime adaptation exploits ML predictors to preserve the timeliness of the HMT, the proposed adaptation action is asynchronously subject to formal verification to provide stakeholders with an assessment of the agents’ decisions and of the overall HMT trustworthiness. The validation testbed is a case study from the healthcare robot domain. The optimized versions of 1000 different mission configurations lead to a statistically significant estimated increase of HMT performance. The asynchronous SMC formal verification confirms that 75% of the optimized configurations increase the HMT performance.