Overview
A 2D simulator built to train driving policies for the autonomous racecar offline and transfer them to the real car. It models bicycle dynamics with actuator delays, simulates the car's LiDAR, IR, and IMU, and is calibrated against logs recorded off the physical car so that what a policy learns in simulation still holds on hardware.
The Problem
Tuning a controller on the physical car is slow: every experiment costs track time, a charged battery, and a car that has to survive the mistake. Moving that work into simulation only helps if the simulation is honest, because a policy trained against idealized dynamics learns to exploit exactly the parts the simulator got wrong. So the simulator was fit to the car rather than the other way around: its dynamics parameters were calibrated against real IMU logs, and the controller inside the loop is a byte-exact port of the firmware that runs on the car.
What I Did
Dynamics and sensor models
Bicycle-model dynamics with actuator delays, plus ray-cast LiDAR and IR sensor models and a filtered IMU, so a policy sees the same kind of noisy, delayed observations it will get on the real car.
System identification with CMA-ES
Fit 6 motor, friction, and latency parameters against real car IMU logs using CMA-ES, which surfaced a 187 ms steering transport lag between command and servo response.
Byte-exact firmware in the loop
The PD controller in the simulator is a byte-exact port of the car's firmware, so training happens against the same control law that ships on the hardware rather than an approximation of it.
PPO residual policy
A PPO residual-policy pipeline learns a correction on top of the PD baseline, trained through a 3-phase multi-agent racing curriculum that builds from solo laps to racing against other cars.
Calibrated Parameters
The six values CMA-ES fit against recorded IMU logs from the physical car:
- Motor lag time constant,
0.336 s: first-order lag, the time to reach about 63% of commanded speed - Linear drag,
21.82 N·s/m: velocity-proportional drag - Rolling resistance,
0.170: dimensionless rolling resistance coefficient - Kinetic friction,
0.184: tire grip, which governs slip in turns - Steering transport lag,
0.187 s: steering command to servo response - Throttle transport lag,
0.015 s: throttle command to motor response
The two transport lags are the ones that matter most for control: steering answers a command more than 12x slower than the throttle does, which is exactly the kind of asymmetry a policy trained on idealized dynamics would never learn to anticipate.
Demo
Gallery
Results
- A calibrated simulator rather than a guessed one: 6 motor, friction, and latency parameters fit to real IMU logs with
CMA-ES, rather than hand-tuned until the sim looked plausible. - The latency terms are what the reality gap was hiding: a 336 ms motor lag, a 187 ms steering transport lag, and a 15 ms throttle transport lag, so steering answers a command more than 12x slower than throttle does.
- The friction terms fit from the same logs (21.82 N·s/m linear drag, 0.170 rolling resistance, 0.184 kinetic friction) are what make the simulated car coast and slip the way the real one does.
- Residual PPO policies trained against the same PD control law that runs in firmware, through a 3-phase multi-agent curriculum.
- Honest limitation: the model is 2D, so it captures friction and slip in the plane but not chassis roll, weight transfer, or suspension, and it is only as faithful as the driving regime the calibration logs covered.