Automated Test Process for Highly Configurable Software-Defined Mobility Systems in CI/CD

In the pursuit of customizability and evolvability of vehicle functions, manufacturers shift towards software-defined vehicles (SDV) to enable flexible customization and over-the-air updates. This results in multiple variants and versions of a vehicle model. While shifting to SDVs adds value and flexibility for customers, manufacturers struggle with homologating new and updated functionality because existing testing processes do not scale for high-frequency release cycles that limit available testing resources. Overcoming this challenge by using a coherent test process designed for testing variant-rich systems in continuous evolution will be one of the key enablers for the future development of SDVs. This paper presents an innovative end-to-end pipeline for efficient and comprehensive testing of variant-rich vehicle functionality tailored to an application in continuous development. Our transferable test pipeline employs sample-based variant selection, a software-in-the-loop environment for executing selected variants, and scenario-based testing using mutation-based scenario fuzzing. A central test controller is responsible for managing the process. We evaluate our test pipeline with regard to its fault detection capability, using the YOLO object detection algorithm as a variant-rich test object in the Carla simulator. Our results show that the testing process outperforms random variant sampling and scenario mutation in detecting faults.