Model-Agnostic Uncertainty-Aware Semantic Segmentation with Conformal Risk Guarantees for Scene Understanding

Abstract

Accurate and reliable scene segmentation is a fundamental requirement for autonomous navigation systems operating in open and dynamic environments. As these systems increasingly rely on data-driven perception modules, their safety and operational robustness hinge on well-calibrated uncertainty estimates that can support explicit control of prediction errors through conformal calibration. Most existing uncertainty-aware segmentation approaches remain architecture-specific and are not evaluated under a common uncertainty-and-calibration protocol across distinct segmentation architectures and datasets. This work introduces a model-agnostic conformal segmentation pipeline that enables operationally meaningful, calibration-based error control in real-world deployments. The proposed framework treats segmentation networks as black boxes and operates on per-pixel class probabilities that are fine-tuned through evidential deep learning (EDL) to decompose aleatoric and epistemic uncertainties. We then apply pixel-wise, class-conditional split-conformal calibration to derive acceptance thresholds for user-defined target error rates. We instantiate the pipeline with DINOv2, Mask2Former, and SegFormer and evaluate it on a newly collected Lisbon street scene (LiSS) dataset; additional cross-dataset results on COCO, using a restricted set of safety-relevant classes, are reported in the appendix. Results show architecture- and class-dependent in-domain uncertainty-error alignment and indicate that dataset shift weakens uncertainty-based filtering and conformal risk control. This motivates continuous monitoring and recalibration as a practical requirement for trustworthy segmentation in safety-critical navigation.