Article by: Antonio Rubio, Project Engineer, Braking Systems in Applus IDIADA
Part One of this series can be found here.
The field of artificial intelligence (AI) has made significant progress in recent years, with applications ranging from natural language processing to computer vision. In recent years, Applus IDIADA Brakes department has presented several studies about artificial intelligence application for detection of brake noises. In this paper, Applus IDIADA presents the research done in this area, but focusing on the development of an AI model for predicting subjective ratings for squeal brake noises based on objective measurements collected through the instrumentation in a typical Brake Noise Durability programme. Subjective ratings are based on human opinions and can be challenging to quantify. Objective measurements, on the other hand, can be objectively quantified and provide a more reliable basis for prediction.
The first part of the article introduced the data processing. This second part, on the other hand, focuses mainly on the AI model creation.
Data cleaning (outliers)
Outliers were removed from the training dataset to ensure more objective learning. Outliers are defined as the values over 1.5 interquartile range (IQR) from the third quartile/75th percentile and below 1.5 interquartile range (IQR) from the first quartile/25th percentile.
This criterion has been used to remove outliers in the variables “Max mic” and “Mic freq” (marked inside the red circle in the following figure 3 and 4).
Figure 3 Outliers marked inside red circle “Max mic”
Figure 4 Outliers marked inside red circle “Mic freq”
Artificial Intelligence Model Creation
Background
The algorithm is trained on the selected objective measurements and corresponding subjective ratings to learn the relationship between the two. The model was evaluated using several metrics, including accuracy, to determine its performance in predicting subjective ratings based on objective measurements.
Training
After the exploratory data analysis (EDA) and review with IDIADA Brakes department, a complete set of noise events with subjective rating from a selected highly skilled driver is used with main objective variables recorded. The total number of noise events selected for the reference driver were 8751, from several years of testing. Data has been split in three parts: 70%/6125 noise events for training, 20%/1734 for test and 10%/892 for validation. Number of noise events per rating were distributed according to the frequency in which they appear in the original data set. Distribution of subjective rating in the data set used for training is as follows (table 3):
Table 3 Distribution of subjective rating in the data set used for training
Final model developed
Several models were implemented using different techniques and criteria. The final model is presented with the best results achieved during the validation.
A combination of two trained models is proposed to get notable accuracy of the results: a classification model and a regression model. If the classification probability exceeds a defined threshold, the classification result is chosen; otherwise, the regression result is rounded to an integer.
For both classification and regression model, an XGBoost (Extreme Gradient Boosting) is used. It is a machine learning library that is used for both classification and regression tasks. It is an implementation of gradient boosting, which is an ensemble learning technique that combines typically decision trees, to create predictive models.
Classification model
In the classification model, an XGBoost Classifier is employed to predict the rating as a category. The rating is treated as a categorical variable. In this case, the prediction is a category (rating 9, rating 8, rating 7 etc.) adding a “confidence” of the model in the predicted rating. In this model, the output is the “confidence” of the prediction for each rating/category. A criterion to select the category according to the confidence in the prediction has been defined. An example output of classification model is shown in Table 4.
Table 4 Example output prediction of the classification model
This model has the drawback of potentially giving a prediction with no clear category with high confidence for a specific rating. In addition, it could give high prediction to different ratings that are not consecutive. In the case of the example shown in table 4, ratings with higher confidence are rating 9 and rating 7. Results during test phase of the model could be checked in figure 5.
Figure 5 Classification model predictions
Regression model
In the regression model, an XGBoost Regressor is employed to predict the rating as a numeric variable. The prediction is a numerical value with a decimal. The criteria to select rating above or below the prediction has been defined as rounded. Results during test phase of the model could be checked in figure 6.
As an example to clarify the concept, the output of the regression model could be “8.4”, so the predicted rating will be rating 8. In the case of an output of “8.5”, predicted rating will be rating 9.
Figure 6 Regression model predictions
Overall model developed
After implementation and analysis of several models, strategies and in order to maximize the combination of two trained models, a classification model and a regression model working inside the model are proposed. First, the classification model performs a prediction if the confidence exceeds a defined threshold. Classification result is chosen; otherwise, the regression result is rounded to an integer (figure 7 and table 5). The threshold is calculated as the optimal for the current data set.
Figure 7 Scheme modelization of predicted rating
Table 5 Subjective rating criteria for each model
About Applus IDIADA
With over 25 years’ experience and 2,450 engineers specializing in vehicle development, Applus IDIADA is a leading engineering company providing design, testing, engineering, and homologation services to the automotive industry worldwide.
Applus IDIADA is located in California and Michigan, with further presence in 25 other countries, mainly in Europe and Asia.
