ADELANTO, Calif. — The following is the third of a three-part series produced by Carlos Lujan Tutusaus, Senior Manager Homologation, Commercial Vehicles, Automated Driving & Vehicle Dynamics in Applus IDIADA and Oriol Flix Viñas, Homologation Engineer, Commercial Vehicles, Automated Driving & Vehicle Dynamics in Applus IDIADA.
Link to Part One
Link to Part Two
This article describes the main future trends with regards to the new assessment and testing methodologies for the type-approval of vehicles and their systems.
The first part of the article introduced the context, as well as the challenge of automated vehicles type-approval; the second part, on the other hand, presented the rulemaking structure.
4. VMAD and the new Assessment and Testing Methodologies
As stated in the previous section of this article, one of the topics of the safety vision of the framework document is the validation for system safety. As it was earlier introduced, the automated and autonomous technologies require a new approach to the validation methodology, different from the classical tests for a prototype under repeatable conditions.
The seed for this new technology is the “three pillar approach”, firstly introduced in the GRVA discussions upon initiative from OICA.
The seed for this new technology is the “multi-pillar approach”, firstly introduced in the GRVA discussions upon initiative from OICA in 2019 [2]. According to this approach, the use of different tools is required in order to guarantee the safe market introduction of automated and autonomous vehicles:
a) Audit / assessment:
- Understand the system to be certified
- Assess that the applied processes and design/test methods for the overall system development (HW and SW) are effective, complete and consistent
- Assess system’s strategies/test performance to address (multiple) fault-conditions and disturbances due to deteriorating external influences; vehicle behavior in variations of critical scenarios
- Simulation: Test parameter variations (e.g. distances, speeds) of scenarios and edge-cases that are difficult to test entirely on a test track
b) Physical CertificationTests:
- Assess critical scenarios that are technically difficult for the system to cope with, have a high injury severity (in case the system would not cope with such a scenario) and are representative for real traffic
- Compare with critical test cases derived from simulation and validate simulation tools
c) Real World Test Drive:
- Assess the overall system capabilities and behavior in non-simulated traffic on public roads and show that the system has not been optimized on specific test scenarios
- Assess system safety requirements like e.g. HMI and ODD
- Assess that the system achieves a performance comparable to an experienced driver
The multi-pillar approach has been further developed under the workframe of VMAD, being renamed as NATM: New Assessment and Testing Methods. For such purpose, and additional level of subgroups, dealing with different specific topics was also created (figure 4):
During the 12th Session of GRVA (January 2022), VMAD presented a proposal for a second iteration of the Master Document on NATM [3], which defines the principles of such methodology on the basis of five pillars:
1. Simulation/virtual Testing
It uses different types of simulation toolchains to assess the compliance of an ADS with the safety requirements on a wide range of virtual scenarios including some which would be extremely difficult if not impossible to test in real-world settings. The aspect of credibility of simulation/virtual testing is included in this topic.
2. Track testing
It uses a closed-access testing ground with various scenario elements to test the capabilities and functioning of an ADS.
3. Real world testing
It uses public roads to test and evaluate the performance of ADS related to its capacity to drive in real traffic conditions.
4. Audit/assessment procedures
They establish how manufacturers will be required to demonstrate to safety authorities using documentation, their simulation, test-track, and/or real-world testing of the capabilities of an ADS. The audit will validate that hazards and risks relevant for the system have been identified and that a consistent safety-by-design concept has been put in place. The audit will also verify that robust processes/mechanisms/strategies (i.e., safety management system) that are in place to ensure the ADS meets the relevant safety requirements throughout the vehicle lifecycle. It shall also assess the complementarity between the different pillars of the assessment and the overall scenario coverage.
5. In-service monitoring and reporting
It addresses the in-service safety of the ADS after its placing on the market. It relies on the collection of fleet data in the field to assess whether the ADS continues to be safe when operated on the road. This data collection can also be used to fuel the common scenario database with new scenarios from the field and to allow the whole ADS community to learn from major ADS accidents/incidents.
In order to guarantee the efficiency of those pillars, they need to be supported by a scenario catalogue, descriptions of real-world driving situations that may occur during a given trip, will be a tool used by the NATM-pillars to validate the safety of an ADS.
This new approach leads consequently to a change in the interaction between Technical Services and vehicle manufacturers. As per the traditional approach (figure 1), it was limited to very late stages of the process, once the vehicle or system had already been validated by the manufacture. However, the new approach requires the type-approval process to be started way in advance, so that the development methodology can be evaluated before the vehicle is ready to be assessed. (figure 5).
5. NATM vs Other Methodologies
Type-approval of vehicles and systems is normally the latest step in the life-cycle of the vehicle development. Previously, the manufacturer may have performed its internal validation tests, in order to guarantee the safety of the vehicle, but also some aspects which are not part of the areas of interest of type-approval, such as the feeling or the comfort of the vehicle.
This may have leaded to different approaches to the testing and assessment methods, depending on the stage of development of the vehicle. However, there is a trend for the harmonization of such methods, and it is becoming frequent for manufacturers to use a similar approach for the validation of their vehicles. Additionally, other evaluation frameworks, like EuroNCAP are introducing similar methodologies.
Efforts for harmonization of these methodologies can also be found in EU-funded projects like HEADSTART, which aimed to define testing and validation procedures of Connected and Automated Driving functions including key technologies such as communications, cyber-security and positioning. The tests will be in both simulation and real-world fields to validate safety and security performance according to the key users’ needs. Those key users included:
a) Type-approval rule makers
- New CAV type-approval regulation
- CAV safe market introduction
- Digital driving license
b) EuroNCAP
- New official assessment protocols
- User acceptance
- Safety-aware sales growth
c) Vehicle manufacturers
- New development strategies
- New CAV functions enabler
- Cost and time-to-market reduction
6. Conclusion
The introduction of automated vehicle technologies introduces a series of challenges to vehicle industry stakeholders, and type-approval is an important one. Guaranteeing that a vehicle is safe to be placed in the market is essential, but it shall be balanced with a certain flexibility to allow the market introduction in a pragmatic and cost-effective manner.
The use of innovative methodologies is thus required, as the traditional methods have proven not to be valid for vehicles able to perform dynamic driving tasks on their own.
Rulemaking forums have already stepped forward and developed an innovative methodology that will allow the type-approval of those vehicles, but as a side-effect, all the involved stakeholders will need to adapt themselves to the new tools, time span and methods introduced by NATM.
Finally, there is also a clear trend for harmonization of the validation methodology along the automotive industry, which reflects the effort to optimize resources and time.
About Applus IDIADA
With over 25 years’ experience and 2,450 engineers specializing in vehicle development, Applus IDIADA is a leading engineering company providing design, testing, engineering, and homologation services to the automotive industry worldwide.
Applus IDIADA is located in California and Michigan, with further presence in 25 other countries, mainly in Europe and Asia.
