How to use this calibrator on carla simulation.

[ahsan155]

I wanted to know if this calibrator has been used on carla simulation. If so, then what configuration was used on the carla vehicle while collecting data. Currently, I am trying to do steering wheel calibration. I tried to launch the steering calibrator as it is. but It wasn't accepting any data. so, I edited the file data_collection_steer.py, so It accepts data coming from carla.

self.declare_parameter('max_velocity', 10.0)
self.declare_parameter('throttle_threshold', 40)

I collected data for the steering range 0.04 - 0.1.

I set the carla vehicle to autopilot and it kept driving. but on autopilot mode, data collection takes too long. so I placed the vehicle on an empty plane and set vehicle throttle in a range (0.0, 0.4) for low throttle and (0.401, 0.5) for high throttle. I set steer in range 0.04 - 0.1. This is the carla code, I use to drive the vehicle
actors = world.get_actors()

# Find the Tesla Model 3 (assuming only one Tesla Model 3 in the scene)
tesla_model3 = None
for actor in actors:
    if actor.type_id == 'vehicle.tesla.model3':
        tesla_model3 = actor
        break

if tesla_model3 is None:
    print("Tesla Model 3 not found in the scene.")

ctr = 0
# Apply control repeatedly
try:
    while True:
        # Generate random throttle and steering values
        throttle_value = random.uniform(0.0, 0.4) 
        steering_value = random.uniform(0.04, 0.1) 

        # Create a vehicle control command
        control = carla.VehicleControl()
        control.throttle = throttle_value
        control.steer = steering_value

        
        if ctr % 20 == 0:
            control.throttle = 0
        else:
            control.throttle = throttle_value
        ctr+=1
        
        
        # Apply the control to the Tesla Model 3
        tesla_model3.apply_control(control)

        print(f"Applied control: throttle = {throttle_value:.2f}, steer = {steering_value:.2f} to the Tesla Model 3")

        # Sleep for a short duration before sending the next command
        time.sleep(0.1)  # Adjust the sleep time as needed

except KeyboardInterrupt:
    print("Stopped by user")

Driving like this in unrealistic. Could this be an issue? With this setting, vehicle tire kept rotating right. In CARLA, the vehicle steering tire rotation (steering angle) value range typically spans from -1 to 1. but steering threshold in the code doesn't cover for negative value.

This is the data file
steering_01.csv
If I visualize the data, it looks like this

I ran neural_network_steer1.py with the data. It returned the result:

[INFO] [1720047302.689904222] [neural_network_steering1]: Mean Squared Error on Test Data: 0.5248184204101562
[INFO] [1720047302.690895569] [neural_network_steering1]: Mean Absolute Error on Test Data: 0.5239980816841125
[INFO] [1720047302.691458616] [neural_network_steering1]: Root Mean Squared Error on Test Data: 0.7244435246519608
[INFO] [1720047302.692560871] [neural_network_steering1]: R-squared (R2) Score on Test Data: 0.16559821367263794

For this steering range, should MSE score get near 0 and R2 near 1? What is the best performance that can be achieved?
Did I do something wrong in data collection that the accuracy is so low?

Also, there are 5 different steering map that can be generated? Which one should I use. My end goal is to drive carla vehicle using Autoware which requires one steer_map.csv.

If you are familiar with carla then I will share the code I used to published data on different ROS topic for calibrator to subscribe to.

I really appreciate any help or suggestion you can provide.