AAA tests driver-monitoring systems, finds many lacking – Ars Technica

If you’re going to install an advanced driver assistance system to let the person behind the wheel go hands-free, then you really ought to include a camera-based driver-monitoring system.

That’s a key finding from a new study by the Automobile Association of America, which recently put a number of new cars to the test in order to find out whether monitoring systems are any good at preventing drivers from becoming disengaged. AAA also tested how easily those systems could be circumvented.

AAA tested four different cars fitted with an ADAS that qualified as “SAE level 2,” meaning that, when activated, the system would steer the car and maintain its speed (slowing if the car in front slows), with the human in the driver’s seat being required to provide situational awareness at all times. (Hence, the DMS to make sure that happens.)

The cars chosen were a 2021 Cadillac Escalade with Super Cruise, which uses a driver-facing infrared camera; a 2021 Subaru Forester with EyeSight and DriverFocus, which also includes a driver-facing infrared camera; a 2021 Hyundai Santa Fe with Highway Driving Assist, which uses a steering wheel torque sensor as an indirect DMS; and a 2020 Tesla Model 3 with Autopilot that does have an interior camera, albeit one not used for any kind of driver monitoring.

The cars were each tested over a 24-mile (38.6 km) test route on California Highway 241, picked for consistent traffic movement at or near the posted 65 mph speed limit. The cars were tested during daylight and night conditions to see whether light levels affected a DMS’s ability to work, and AAA noted that it fully complied with California laws during the testing, that the “driver disengagement modes consisted of simulated inattention,” and “that at no point were electronic devices utilized by test drivers.”

AAA tested two different disengagement scenarios: one where the driver kept their head up and facing the road but their eyes looking down (scenario A), or with the head and gaze both directed down toward the center console (scenario B). In both scenarios, the driver had their hands off the wheel.

For starters, there was no difference in the performance of any of the systems between day and night. As you might expect, the camera-based DMSes were far more effective than indirectly measuring engagement via a torque sensor on the steering column.

In scenario A, the average distraction time was 7.7 seconds for the Cadillac and 8.1 seconds for the Subaru. By contrast, the distraction time in the Tesla was 37.7 seconds and 78 seconds for the Hyundai. AAA noted that the Tesla was consistent in providing alerts at between 37-39 seconds, but that alert timing would vary in the Hyundai “from approximately 2 minutes to less than 20 seconds from the point at which the test driver removed their hands from the steering wheel.”

The results were similar for scenario B. Here, the Cadillac driver averaged 6.7 seconds of distraction, and the Subaru just 4.1 seconds. But the Tesla allowed 37.8 seconds of distraction, and the Hyundai 74.4 seconds.

Next, AAA wanted to see whether the monitoring systems could be circumvented “via periodic head/eye movements and/or steering wheel input with the ADA system engaged.” However, it did not employ any tools like the “autopilot buddy” that’s marketed in the Tesla community.

The testers found that it was possible to actively circumvent attention warnings in all four cars. The Subaru was the most resistant to circumvention, but drivers were able to increase the average distraction time to 79.4 seconds. And for the Cadillac, this stretched to 191 seconds. But the indirect driver-monitoring systems in the Hyundai and Tesla were far easier to fool—the distraction times in these cars could be stretched to 334.7 seconds (Hyundai) and 342.2 seconds (Tesla).

“Regardless of brand names or marketing claims, vehicles available for purchase today are not capable of driving themselves,” said Greg Brannon, director of AAA’s automotive engineering and industry relations. (Wise words that The New York Times’ Farhad Manjoo would have done well to remember when he tested Super Cruise.) “Vehicle technology has the potential to improve roadway safety, but the last thing we want are ineffective features in the hands of uninformed or overconfident drivers,” Brannon said.