An “over reliance” on Tesla’s driverless technology played a major role in a fatal accident in the US last year, an independent investigative body has found.
In May last year, a Tesla Model S sedan operating in Autopilot mode – a semi-autonomous driving feature that handles most of the steering and speed on freeways – crashed into a truck that had turned in front of it without giving way, killing the car’s 40-year-old driver, Joshua Brown.
The National Transportation Safety Board (NTSB) presented its investigation into the crash this week, finding that the Tesla Autopilot feature had to bear some of the blame for the accident because it had allowed the driver to become over-reliant on it and not pay proper attention to the road.
But the board ruled that the crash was ultimately down to human error, with the truck driver not yielding when turning onto the highway, and the Tesla driver not watching the road to brake in time.
The Autopilot feature is only meant to be used on well-marked, protected highways with no intersections, and was not intended to be used where the crash took place, or to have detected the turning truck.
But the NTSB found that Tesla’s feature had allowed the driver to become over-reliant on the autonomous function, lulling him into a false sense of security that made him unable to stop the car.
“The combined effects of human error and the lack of sufficient systems controls resulted in a fatal collision that should not have happened,” NTSB chairman Robert Sumwalt said.
The investigation found that in the 42 minutes prior to the fatal crash, the Tesla car had been in auto-steering mode for 37 minutes, travelling at about 120 km/h. The car had given the driver visual warnings to place his hands on the steering wheel seven times, along with sound warnings, but continued to drive when he failed to do this.
This is where the system is partly to blame, the NTSB found.
“In this crash, Tesla’s system worked as designed, but it was designed to perform limited tasks in a limited range of environments,” Sumwalt said.
“Tesla allowed the driver to use the system outside of the environment for which it was designed.”
Four months after the accident, Tesla issued an update for its Autopilot system, making it much harder for drivers to ignore warnings to regain control of the car, and automatically turning off the feature if they don’t.
Tesla defends feature
In a statement, Tesla defended the safety of its Autopilot feature.
“At Tesla, the safety of our customers comes first, and one thing is very clear: Autopilot significantly increases safety,” a Tesla spokesperson said.
“We appreciate the NTSB’s analysis of last year’s tragic accident and we will evaluate their recommendations as we continue to evolve our technology. We will also continue to be extremely clear with current and potential customers that Autopilot is not a fully self-driving technology and drivers need to remain attentive at all times.”
The board’s ruling will have broad ramifications for the rise of driverless cars and the companies behind them. While the Autopilot system was being used incorrectly before the crash, a number of companies have been competing against each other to market their offerings as more autonomous than the rest.
As the crash demonstrated, this has led drivers to become overly dependent on the technology, which is not yet ready to be fully autonomous.
In the limbo period while cars with driverless technology are on public roads but still require an attentive driver behind the wheel, tech companies will now have to be increasingly careful with how they are designed, and how this is communicated to drivers.
Brown's family speaks
Joshua Brown’s family issued a statement following the board’s report, saying they support Tesla and believe the crash was down to chance.
“Part of Joshua’s legacy is that his accident drove additional improvements making the new technology even safer,” the statement said.
“Tesla has done extensive research into the accident and how it might have been prevented. Our family takes solace and pride in the fact that our son is making such a positive impact on future highway safety.
“Joshua believed, and our family continues to believe, that the new technology going into cars and the move to autonomous driving has already saved many lives. Change always comes with risks, and zero tolerance for deaths would totally stop innovation and improvements.”