Validation Plot

Description

While Predicted vs Actual provides a point-by-point comparison for each data point Validation plot makes it possible to compare the predicted with the actual values as line plots. The actual and predicts values are plotted versus another variable in your dataset.

Application

There are two typical applications for this function. The first is if you have any sort of series data. That might be time series, frequency series, etc. You typically would plot the output variable versus the column in your data establishing the series (e.g. time or frequency).

The second application is if you want to do a sweep of the output along some parameters. For example, you could plot the drag coefficient of a car’s rear wing along the angle of attack.

It is worth noting that if using a series model, the results in the plot will be downsampled to 10k points, which might not be the same across different models

How to use

• Create the step and assign a tabular dataset to it in the field Data. The function should usually be run with test data unseen by the model during training.
• Select the Modelsfor which you want to create the comparison. The selected models should at least have one input and output in common.
  • If the model calculates uncertainty this will be plotted as a transparent red band around the prediction as well.
• In the X axisfield you can choose any of the columns which are available in the dataset.
  • The step provides a lot of flexibility regarding the selection for the x-axis. Nevertheless, the most reasonable and most helpful results can be achieved if a series variable (like time) is used or if there is a clear and relatively smooth dependency between x-axis and y-axis (=output).
• In the Y axis field you can choose one output common to all selected models.
• If your dataset contains not just data from a single test run you can use the option Column grouping individual tests to identify each of those tests and plot separate lines for each test. The column assigned here should provide a unique identifier for each test.
• Click Apply to run the step and plot the data.

Examples

Plot output versus time

The plot below shows the prediction of the force at the rear wheel for a certain driving manoeuvre. In this example the prediction of two models is compared with the actual data from measurement.

In this case, the plot shows that both models were successful in accurately predicting the behaviour of a new manoeuvre, which can increase the trust of the user in these models.

Sweep output along another parameter

The example below is about the performance of a rear wing of a car. Validation plot is used for a sweep of the Lift Coefficient along the Angle of Attack. In this case a model with uncertainty was used which is shown together with the model prediction in comparison to the actual data.

In this case, the user can see that although the predictions are not perfect, they are really accurate, and true values are always found in the range of uncertainty.