A major transformation is underway in automotive safety with the introduction of FMVSS No. 127. Starting in 2029, the National Highway Traffic Safety Administration (NHTSA) will require all passenger cars and light trucks in the United States to be equipped with automatic emergency braking (AEB) systems that meet strict and measurable performance criteria for speed, distance, and reliability.

Additionally, there’s the recent transformation of the Euro NCAP 2026 regulation, a comprehensive four-stage vehicle safety framework, with the introduction of a robustness layer to ensure the performance of the perception stack.

Of course, these standards are not just guidelines — they are enforceable requirements that will define market access for original equipment manufacturers (OEMs) well into the future.

The stakes are considerable, as compliant AEB systems are now projected to save 360 lives and prevent 24,000 injuries each year. Meeting these expectations, however, demands unprecedented levels of system robustness and validation rigor.

To address this challenge, Valeo’s camera experts, in collaboration with Ansys, part of Synopsys, enhanced traditional validation workflows by introducing an additional layer into software-in-the-loop (SiL) and hardware-in-the-loop (HiL) simulation processes. This advanced approach strengthens confidence in system performance and helps ensure that advanced driver assistance systems (ADAS) consistently meet the demanding requirements of new regulations.

Perception Errors: A Critical Challenge in ADAS Performance

Perception errors happen when an ADAS incorrectly interprets its environment and predicts a potential hazard where none exists resulting in unnecessary interventions such as braking or deceleration. These types of events are essentially false positives within the perception–prediction–planning chain that governs system behavior.

However, in modern ADAS architecture, such errors rarely originate from a single sensor. Instead, they often arise from sensor fusion processes where inconsistencies or misalignments between sensor output, due to the different physics involved, can lead to incorrect system-level decisions.

Sensors and perception algorithms play an increasingly critical role in how we drive. As such, the need to rethink processes and ensure the appropriate level of detail must be considered at every step of the design phase. In addition, the ability to iterate between system-level and component-level analysis at an early stage of vehicle development is crucial.

To mitigate these perception challenges, automotive manufacturers rely on the extensive calibration and tuning of ADAS software in consideration of factors such as windshield properties, bumper materials, and vehicle dynamics. This activity helps them balance system sensitivity with the robustness needed to minimize both missed detections and false positives.

However, this iterative process also increases development complexity, ultimately driving up costs and extending the time leading up to start of production (SOP). In many instances, further adjustments are required via post-production reflected in over-the-air (OTA) software updates or vehicle recalls.

Optics-in-the-Loop Brings Realism to ADAS Camera Validation

One approach to mitigating perception errors is the upfront validation of camera-based ADAS through simulation, combining SiL and HiL testing. It is a widely adopted framework that often relies on simulation platforms derived from gaming engines.

While efficient, such environments typically do not fully reproduce the physical propagation of light from outside the vehicle to the camera’s complementary metal-oxide-semiconductor (CMOS) sensor. Consequently, some perception issues can go undetected until late-stage development.

There is a clear explanation for this limitation. In conventional SiL setups, perception algorithms are evaluated using synthetic or prerecorded images, focusing on tasks such as pedestrian detection, lane recognition, and distance estimation. However, these approaches generally assume ideal input images, neglecting the optical distortions introduced along the real optical path, particularly through the windshield.

More advanced SiL techniques aim to address this gap by modeling the physical behavior of light within the vehicle. This includes preserving spectral fidelity of vehicle coatings, lenses, and the windshield for a more accurate response across relevant wavelengths. When properly implemented, such physics-based simulations can help detect perception issues early in the development cycle.

During HiL testing, the focus shifts to system-level validation using actual camera hardware. In this context, simulated or recorded scenes are injected into the perception stack, often running on the target system-on-chip (SoC) at the required real-time frequency. Now, achieving real-time performance imposes constraints as camera simulation may need to compromise on full physical accuracy in favor of execution speed, particularly regarding detailed optical effects like lens coatings or complex light interactions.

Neither traditional SiL nor HiL testing fully captures what happens before light reaches the camera sensor. That optical path from the external environment, through the windshield, and into the camera is often simplified or overlooked. This is significant because the windshield can, in fact, introduce a range of subtle effects, including internal reflections between glass layers, diffraction from heating wires, contrast degradation due to dust or microdroplets, or glare in backlit conditions.

Although these effects are often barely perceptible to the human eye, they can meaningfully alter how the camera represents the scene internally. When perception algorithms are trained and validated without accounting for these factors, they may behave unexpectedly in real-world conditions, ultimately leading to perception errors such as unnecessary braking events.

To help overcome these limitations, a unique complementary test approach was proposed that considers multi-spectral energy information all along the ray propagation in the scene as well as ray’s multiple bouncing management (the computational management of electromagnetic wave paths reflecting off of multiple surfaces before reaching a radar receiver).

This approach, supported by Valeo’s product expertise, enabled a successful end-to-end camera digital twin incorporating optics-in-the-loop (OiL).

Seeing Clearly With OiL and a Digital Twin

OiL encompasses all of the optical control systems needed to accurately represent the optical part of the camera sensor involved in an ADAS. Instead of starting from an assumed perfect system described in a datasheet, the method begins with a physically accurate modeling of the camera system including the housing, the vehicle’s windshield, and any other impactful components.

None of the classical SiL or HiL simulation explicitly models what happens before the image reaches the camera sensor as it relates to the optical path, or the way light passes from the real environment, through the windshield, into the camera. This omission matters because the windshield can introduce any number of errors, such as:

Ghost reflections between glass layers
Shadowing patterns caused by heating wires
Contrast loss due to dust, mist, or microdroplets
Glare effects in backlit situations

The classic validation cycle starts with model-in-the-loop (MiL), moves to SiL, then to HiL, and ends with actual vehicles on the track at the proving grounds or on open roads in the real world. According to UNECE New Assessment/Test Method for Automated Driving (NATM) guidelines, each of the steps in this multi-pillar framework approach contributes to specific insights, detection, and test coverage.

Integrating optical controls as an optical system digital twin into an SiL/HiL validation chain makes the system validation process more predictive, reliable, and efficient. This is accomplished using feedback methods as well as adaptive optics to improve system performance.

OiL as general concept can be further classified into two main subcategories:

Full OiL: This approach retains full spectral resolution and exhaustive ray‑bouncing computations, with no simplifications or model reductions. By preserving the complete physical modeling of optical interactions, it delivers maximum accuracy and serves as the reference baseline for validation. This level of fidelity does not yet allow for close-to-real-time simulation.
Real-time OiL: This solution is built on parametric representations and reduced‑order modeling techniques to meet the stringent real‑time computation requirements of HiL simulations. The method enhances computational efficiency by optimizing spectral sampling strategies and ray‑tracing/bouncing algorithms, enabling high‑speed execution while preserving physical accuracy.

These two categories do not work in isolation. They are meant to connect with each other. Implementation of this approach remains in the early stages for now, but is promising in the analysis of high-fidelity phenomena.

Zooming in on What’s Important

Recently, Valeo collaborated with Ansys to enable full OiL by integrating advanced optical simulation capabilities into a camera validation framework to strengthen virtual validation and assessment for more robust ADAS.

Together, the engineering teams built a high-fidelity optical digital twin of Valeo’s Smart Safety 360 EQ4 front camera installed behind the windshield.

In the creation of this digital twin, it was possible to reproduce the interactions of the front camera within its operating environment, including:

The imaging characteristics of the camera lens (resolution, distortion, and transmission)
The sensor’s physical response (quantum efficiency, noise, and gain curves)
The impact of the windshield’s laminated glass structure and internal reflections on perception
The optical signature of the heating wires embedded in the windshield and positioned in front of the camera (field of view occultation and thermal load effects on windshield optical properties)
Contrast changes caused by contamination on the windshield’s glass surface

The model has been validated against real measurements from both test-bench and vehicle testing. This ensures the simulated images are not only visually similar, but also quantitatively consistent with reality. As a result, the disruptive optical phenomena can be analyzed with greater confidence.

As driving automation advances, the need for and emphasis on reliability in the development of automotive perception systems will certainly continue to grow. To this end, ADAS camera systems must be evaluated not only on how they interpret the world, but on how they actually see it to prevent dangerous incidents in the future.

OiL Brings ADAS Safety and Development Efficiency Into Sharper Focus

The introduction of OiL during validation in a simulation environment enhances system safety, reduces development risk, and contributes to more predictable and trustworthy vehicle behavior where it matters most. It strengthens the ADAS development process by making perception validation more realistic from the start. This leads to several benefits, such as:

Earlier detection of conditions that could cause perception errors
Reduced debugging during road-testing phases
More accurate communication with OEMs regarding windshield and installation specifications
Better robustness of perception algorithms across lighting and weather conditions
Lower development cost through fewer late-stage test iterations

In short, OiL improves confidence in camera perception before the vehicle ever reaches the road.