Artificial intelligence ensures traffic safety in the San Bernardino road tunnel

Can artificial intelligence make road tunnels safer and more efficient? The Vehicle Hotspot Detection Systems (VHD) from SICK at the northern and southern entrance of the San Bernardino tunnel answer this question with a resounding “Yes!”. Using deep learning, they have been trained to distinguish hazardous overheating hubs on vehicles from hot but not dangerous mufflers. The result: higher safety with fewer false alarms and optimal traffic flow.

“The VHD systems from SICK are used to detect trucks and buses with overheated surfaces, i.e. surfaces at risk of combustion before they drive through a tunnel, onto a ferry, into a loading terminal, or into a traffic checkpoint,” explains Lukas Wallimann, Product Manager VHD at SICK in Buochs, Switzerland. The system perform this task by combining images from infrared cameras with the measurement data from 2D LiDAR sensors.

The VHD system from SICK detects overheated surfaces.

This enables the system to measure temperatures on the vehicle and determine its dimensions in order to create a 3D model on which individual vehicle areas such as the wheels, motor, exhaust or load compartment can be automatically identified. “These kinds of systems have been installed throughout Europe during the last decade, including on the Mont Blanc tunnel and on the Gotthard tunnel,” says Christoph Gilgen, Customer Project Manager at SICK. That VHD systems have proven in practice to be a reliable safety measure against vehicle fires in tunnels is confirmed by Italo Broggini at the Bellinzona branch of the Swiss Federal Roads Office (ASTRA): “The thermal portal on the San Bernardino helps us remove overheated vehicles from the traffic thereby contributing to tunnel safety.”

The thermal portal on the San Bernardino helps to remove overheated vehicles from traffic, thus contributing to tunnel safety.

“Room for improvement” for the alarm quality

So they’re effective, the VHD systems. But to make them even more efficient, it was necessary to further improve the quality and accuracy of the alarms. This was an essential prerequisite for the San Bernardino project because the rescue service for the tunnel is only located at the southern portal. If the thermoportal raises an alarm at the northern entrance to the tunnel, which is about seven kilometers (4.3 miles) away, it will take emergency personnel some time to get to the hazardous vehicle. False alarms, e.g. triggered by a hot muffler, therefore lead to unnecessary deployments of the rescue service. But how much “room for improvement” for a higher alarm quality do the VHD systems have – and how can this potential be exploited for greater efficiency as well?

Profiling systems

Fully-automatic detection of overheated vehicle parts

Vehicle Hotspot Detection System

Artificial intelligence improves the alarm quality of the VHD

It's all about the system accuracy. How can it be improved? Simply by using more sensors? Or are new algorithms required to enable potential ignition sources on vehicles to be more reliably identified and distinguished? “The solution can be found in artificial intelligence, or more precisely deep learning,” says Roman Schindler, CNN Software Engineer at SICK. This “CNN”, however, has nothing to do with the news channel in the USA, but instead stands for “Convolutional Neural Network” – an artificial neural network that is inspired by biological processes and is predominantly used for the computer processing of image data. A particular advantage of this method is its low error rate, which it was hoped would be confirmed when using VHD systems with CNN algorithms.

Thousands of thermal images were analyzed to keep the error rate as low as possible.

In the so-called "training camp" of the VHD system, thousands of thermal images were analyzed over four months, thereby training the CNN algorithm.

Learning by training, learning from images

The VHD systems spent four months in a “training camp”. “During this time, thousands of thermal images of existing systems were collected, analyzed, manually classified, and then used to train the CNN algorithms,” says Roman Schindler describing the development process. Furthermore, SICK went through all the alarms during the past few months with the captain of the San Bernardino fire and rescue force to filter out the actual false alarms among all of the alarm notifications. This measure was also useful for the CNN algorithm. The new method significantly improved the system’s ability to differentiate between individual areas of the vehicle such as wheels, motor or exhaust. “At the San Bernardino tunnel, where the new algorithm was tested in real-time operation for the first time, a lower false alarm rate established itself,” says Christoph Gilgen. “This is a more than significant improvement compared to the conventional algorithm, which is also still running there at the thermoportal in the background as a backup.”

Overheated vehicles are taken out of service.

CNN algorithm: upgrade of existing portals possible

“Thanks to the good false alarm rate, there are fewer unnecessary deployments, which results in cost savings and increased motivation of the personnel in the event of an alarm,” points out Italo Broggini from ASTRA as a further positive aspect. No wonder then that the operators of existing VHD systems are “fired up” about upgrading their systems. “Any one of these systems can be retrofitted with the CNN algorithms because it only involves a software update,” explains Lukas Wallimann and adds: “Newly ordered VHD systems will be equipped with this upgrade right away. The customer thereby benefits from SICK’s longstanding experience in the thermal measurement of vehicles without having to pay more for the upgrade than the conventional algorithm.”

VHD systems “never stop learning” either

The current release version of the CNN algorithm is not the final one, however, because the system performance will continue to improve with new image data and training cycles. “This means VHD systems never stop learning,” summarizes Roman Schindler. Furthermore, new and related fields of application are always opening up for these systems. For example, SICK is working on a further development of the VHD system for ferry terminals in order to be able to detect hot spots on the refrigeration units on trucks. More efficiency for VHD systems thanks to the new CNN algorithm: The initial work is complete – no end in sight.