How accurate is Bengaluru’s AI traffic enforcement? | Explained

The story so far: In mid-September, a Bengaluru motorist was issued an online traffic violation challan. When he checked the Bengaluru Traffic Police’s e-challan website, he found that he had been issued a challan for pillion riding without a helmet, even though he had no pillion on his scooter. Soon, he realised the reason: he had been wrongly flagged by the BTP’s Artificial Intelligence (AI)-based violation flagging system for the guitar strapped to his back, which was mistaken for a human.

A senior traffic police officer explained to The Hindu that it was a judgement error by the AI camera, part of the BTP’s Intelligent Traffic Management System (ITMS), which subsequently went through a manual review. However, a challan was issued due to human oversight as well.

The incident raised questions and triggered debates online about the accuracy of AI systems and the shortcomings of using AI in smart policing and rule enforcement. The BTP says that striking a balance between AI systems and human intervention yields the best results, while also acknowledging that certain anomalies will occur.

How are challans raised?

The BTP raises contactless challans using three different systems: through ITMS, Field Traffic Violation and Regulation Analytics (FTVR), and through Public Eye.

The ITMS was adopted in Bengaluru in December 2022 to minimise human intervention in issuing challans and to deploy more officers for traffic regulation. The ITMS uses AI and machine learning to automatically detect traffic violations, and the feed is sent to the Traffic Management Centre (TMC), which is the BTP’s head office.

The FTVR system involves traffic police officers capturing pictures of violators or vehicles in violation of traffic rules and raising a challan online. Public Eye is an option for citizens to take pictures and upload them on the BTP application anonymously.

Image of the two-wheeler rider in Bengaluru, who was slapped with a helmetless riding challan after the AI camera for traffic rule enforcement mistook the guitar for a pillion rider. | Photo Credit: The Hindu

What is the accuracy?

Karthik Reddy, Joint Commissioner of Police (Traffic), claimed that the AI system (ITMS) with minimal human intervention has nearly 99% accuracy and that wrong flagging is negligible.

The Hindu reviewed the Actionable Intelligence for Sustainable Traffic Management (ASTraM) data, an initiative of the BTP that uses AI for traffic management, according to which the BTP issues an average of nearly 26,500 challans through contactless means every day, of which over 19,000 challans are generated after identifying violations through Artificial Intelligence (AI) cameras. Of the average 19,000 challans, 25 cases, or 0.13% of cases, are challenged every day.

But the 0.13% challenge rate cannot be treated as a measure of AI accuracy.

Mr. Reddy explained that once a violation is detected by the ITMS, a team at the Traffic Management Centre (TMC) reviews the violation flagged by the system, after which a challan is generated.

Another senior officer, however, explained that after filtering the data for cases rectified by the team manually reviewing the violations, the AI accuracy could drop to about 90%. By the end of 2023, the BTP had identified that the system’s accuracy had dropped to 80%, especially in certain violations such as pillion riding without a helmet, seatbelt violations and signal jumping. However, over the past two years, the accuracy level has been scaled up.