Data Challenges Threatening Autonomous Trucking Safety
The Hidden Data Challenges for Autonomous Trucking Safety
For autonomous trucks, the question may not be how many miles a vehicle has traveled, but whether those miles included the situations most likely to expose a safety problem.
That is the view of Luc Vincent, a special adviser to road-data company Empiric Earth. Vincent argues that conventional autonomous-vehicle testing has a fundamental limitation: Even large test fleets may rarely encounter the unusual, dangerous events that automated driving systems must handle safely.
Empiric Earth is developing a testing framework called Apex, which is intended to provide a standardized safety scorecard for autonomous vehicles. The framework would evaluate systems across a range of specific driving scenarios and combine those results into an overall rating.
The effort addresses a gap in the current regulatory landscape. There is no universally accepted safety test that autonomous vehicles must pass before operating on public roads. For autonomous trucking, that leaves no broadly recognized equivalent of a driver qualification or carrier safety evaluation focused specifically on the software controlling the vehicle.
That gap matters as developers work to expand automated trucking operations and as policymakers consider how autonomous systems should fit into existing commercial-driver and carrier-safety rules. A truck may perform well during routine highway travel, but safe deployment also depends on how it responds to debris, sudden lane changes, flooded pavement, unusual traffic behavior, severe weather and other conditions that may occur only occasionally.
“In the US, fatalities on highways occur only every 100 million miles, thankfully,” Vincent said. “So to experience, to observe some, to maybe see events that are that rare, you need to drive a lot.”
The statistic illustrates the challenge of relying solely on physical road testing. Serious crashes are rare compared with the total number of miles driven, which is a positive outcome for road users but makes statistically precise validation difficult. If an autonomous system is expected to perform at a failure rate comparable to human drivers, developers may need billions or even hundreds of billions of miles to measure that performance with a high degree of confidence.
Current test fleets cannot practically cover every combination of road design, driver behavior, weather and traffic conditions. Physical testing is also expensive and limited by the number of vehicles, available routes and operating hours.
Simulation can expand testing considerably, but Vincent said simulated environments do not fully capture the behavior and conditions found on public roads. Simulations may model erratic lane changes, pedestrians, construction zones or poor weather, but the models are based on assumptions and previously collected information. They can miss the unexpected interactions that occur in real traffic.
Empiric Earth says it has collected approximately 10 billion miles of real-world driving data. The company has used that data to identify what Vincent described as particularly valuable events, including near misses, road debris, flooded streets, jaywalking pedestrians and wildlife crossings.
Those events are important because average driving conditions do not necessarily reveal the weaknesses of an autonomous system. A vehicle can accumulate a large number of uneventful miles without encountering the specific situation that causes it to misread a roadway or make an unsafe decision.
Geography also plays a major role. Vincent pointed to differences in traffic behavior and road design from one city or region to another. Austin’s horizontal traffic signals, Boston’s left-turn patterns and large pedestrian surges around Halloween can all create conditions that may be unfamiliar to a system trained primarily in another location.
For autonomous trucks, regional differences can extend beyond urban traffic. Long-haul routes may include changing pavement markings, work zones, snow and ice, standing water, livestock or wildlife near the roadway and remote stretches where assistance is not immediately available. A system validated in one operating environment may require additional testing before it can be considered ready for another.
The Apex framework is intended to organize that testing by scenario rather than relying on a single mileage figure. A regulator or other agency could use scenario-level results to evaluate whether a particular autonomous system is prepared for deployment in a specific area. The framework is also designed to change as road users and driving behaviors evolve. Vincent cited scooters as one example of a newer road-user category that may need to be included in future evaluations.
The approach reflects a broader distinction between simulated or artificial intelligence and what Vincent called physical intelligence: information gathered by observing how people actually drive in real conditions. Simulation remains useful for training and repeated testing, but real-world data provides the reference point for determining whether a system is responding appropriately to the complexity of public roads.
As autonomous vehicle developers expand testing and more robotaxis appear on public streets, the need for consistent comparisons is becoming more pressing. The same issue applies to autonomous trucks, where deployment decisions will affect drivers, motorists, roadside workers and freight operations.
The central challenge is not simply collecting more miles. It is finding and measuring the rare events that ordinary testing is least likely to encounter. A safety standard built around those events could give regulators and fleet operators a clearer way to judge whether an autonomous vehicle is ready for the roads and conditions where it is expected to operate.
