Coordinating Multi‑Assistant Interaction in Smart Homes: A Front‑End Delegation Approach

Abstract

Smart home environments increasingly integrate multiple domain-specific assistants, yet interaction remains fragmented across heterogeneous interfaces. Users are required to adapt to different systems and interaction modalities, which introduces complexity and reduces accessibility, particularly in scenarios such as active ageing.
This work aims to design a unified conversational abstraction that enables seamless interaction with multiple specialised assistants through a single entry point. The goal is to decouple user interaction from underlying services while preserving modularity and extensibility.
We present the Butler Assistant, a modular conversational framework that orchestrates multiple domain-specific assistants within a unified architecture. The system introduces a routing layer based on a shared registry and supports interchangeable routing engines. Two approaches are explored: a lexical model based on Naive Bayes and a semantic model based on sentence embeddings. The system is evaluated under both in-domain and out-of-domain scenarios.
The evaluation shows that the semantic model improves routing accuracy in domains with limited training data, demonstrating stronger generalization capabilities. In contrast, the lexical model is more sensitive to training data distribution, achieving strong performance in well-represented domains but degrading in low-resource scenarios. Importantly, both approaches exhibit reliable behaviour in out-of-domain conditions, consistently redirecting unsupported inputs to fallback outcomes and avoiding incorrect domain assignments. These results highlight the complementary strengths of lexical and semantic routing and emphasize the role of system-level decision mechanisms in ensuring safe and robust conversational interaction.