Motorcyclists continue to be amongst the most vulnerable road users in the world. According to the 2018 Global Status Report on Road Safety published by the World Health Organization (WHO), they represent more than half of all global road traffic deaths alongside pedestrians and cyclists. The risk associated with riding a motorcycle or scooter is especially high in South-East Asia, where riders of two- and three-wheeled motorised vehicles represented 43 percent of all registered traffic fatalities. The WHO data further suggests that not wearing a helmet can add significantly to the risk of riding a bike or scooter in busy street environments: Head injuries are the leading cause of death and major trauma among those riding two- or three-wheelers.

“Road traffic injuries are the eighth leading cause of death for all age groups. More people now die as a result of road traffic injuries than from HIV/AIDS, tuberculosis or diarrhoeal diseases”. (World Health Organization, 2018 Global Status Report on Road Safety: Summary)

Innovative research led by computer scientist Dr Hanhe Lin from the University of Konstanz and traffic psychologist Dr Felix Siebert from Friedrich Schiller University of Jena in Germany reported in the journal IEEE Access introduces a new deep learning-based method for detecting motorcycle helmet use. Based on a convolutional neural network (CNN), the novel multi-task learning (MTL) approach improves significantly on previous automated detection systems by delivering a much more accurate and efficient means of collecting road safety data associated with helmet use.

Watch a brief video summary here: H. Lin et al.: Helmet Use Detection of Tracked Motorcycles Using CNN-Based Multi-Task Learning, IEEE Access (2020)

The starting point: road safety in Myanmar

“As a traffic psychologist, one of my main interests is to collect accurate data that can be used to increase road safety, which is why I reached out to Hanhe – because he can provide the kind of precise data that we need to design and implement effective education and enforcement policies”, says Dr Felix Siebert, a postdoctoral researcher at the Department of Psychology at the University of Jena. Earlier in 2020, a global conference on road safety hosted on behalf of the United Nations (UN) in collaboration with the WHO resolved to make a joint effort to reduce road traffic deaths and injuries by 50 percent by 2030, intensifying the efforts made during the original Decade of Action for Road Safety 2011-2020.

“49 countries representing 2.7 billion people currently have laws on motorcycle helmet use that align with best practice”. (World Health Organization, 2018 Global Status Report on Road Safety: Summary)

Siebert is a member of the non-governmental Myanmar Organization for Road Safety and has spent some time studying patterns of motorcycle helmet use in the country. “In 2016, a couple of colleagues of mine and I started out with a video-based observation study, which revealed that motorcycle helmet use in Myanmar is relatively low sitting at only 51.5 percent”, he continues. Amongst other things, the study showed that helmet use varies significantly between cities; that it decreases for every additional passenger on the motorcycle; and that female riders are more likely to wear helmets than male riders.

While the study generated useful insights into the situation in Myanmar, it also revealed a major drawback associated with the technology: “Using low-cost cameras in combination with existing CCTV systems may be a cost-efficient way of keeping an eye on whatʼs happening in the streets, but we may be talking hundreds of hours of raw footage here – thatʼs virtually impossible for any human observer to analyse”.
 

Frame-by-frame comparison of the automated detection and motorcycle tracking approaches

Automated motorcycle detection using a frame-based approach. Publication: F. W. Siebert and H. Lin: Detecting motorcyle helmet use with deep learning, Accident Analysis and Prevention (2020), URL: https://doi.org/10.1016/j.aap.2019.105319

1 von 2

Text ausblenden

Automated motorcycle detection and tracking using a CNN-based multi-task learning approach. Publication: H. Lin et al.: Helmet Use Detection of Tracked Motorcycles Using CNN-Based Multi-Task Learning, IEEE Access (2020). URL: https://ieeexplore.ieee.org/abstract/document/9184871. Licensed under a Creative Commons Attribution 4.0 License.

2 von 2

Text ausblenden

The method: automated detection with machine learning

This is where automated detection comes in, which promises major advancements in terms of practicability, efficiency, and accuracy. Siebert and Hanhe Lin, who is a postdoctoral researcher in the University of Konstanzʼs Multimedia Signal Processing Group, first worked together on this for a research project reported in early 2020 in the journal Accident Analysis and Prevention.

“Human observers are able to distinguish between various kinds of information when looking at video footage”, explains Hanhe Lin, who specialises in machine learning and computer vision. “Classifying various kinds of vehicles, telling drivers and passengers apart, or determining whether someone is wearing a helmet or not – humans have no problem with these kinds of very simple tasks. Algorithms, by contrast, donʼt find that kind of work easy to perform”. The main challenge Lin and Siebert had to address as they started looking into automated detection – and, subsequently, into automated tracking systems – in road traffic contexts was to train an algorithm to be as accurate as a human observer.

“To produce helmet use estimates that achieve the same kind of accuracy as human observers, automated detection systems must be able to detect motorcycles, to track them for a few seconds, to register rider numbers and positions, and to account for site diversity”. (Dr Hanhe Lin, postdoctoral researcher at the University of Konstanz)

Using 91,000 frames of video data collected at multiple observation sites across Myanmar, the researchers first trained an algorithm to detect active motorcycles, the number and position of riders, and whether they were wearing a helmet or not – an approach which is based on the laborious labelling of data by hand. “It sounds fancy, but what we actually had to do was go through the hundreds of hours of raw footage first collected during the 2016 study and manually annotate the individual frames”, continues Lin. As the corresponding paper on detecting motorcycle helmet use with deep learning suggests, the algorithm proved accurate within -4.4 percent and +2.1 percent as compared to human observers, with slightly decreased performance for sites that had not been part of the training process.

F. W. Siebert, H. Lin: Detecting motorcycle helmet use with deep learning, Accident Analysis & Prevention (2020)

https://www.youtube.com/watch?v=LmCqwC-UC-I&feature=youtu.be

Motorcycle detection using deep learning: Annotating individual frames manually, Lin et al. trained the algorithm to detect active motorcycles, the number and position of riders, and whether they were wearing a helmet or not.

“However, we soon realized a major limitation of our approach: We could get the algorithm to detect helmet use based on individual image frames, but it was unable to track a motorcycle as it passed from one image frame to the next”, explains Siebert. In other words, while the system did deliver correct average helmet use data through a frame-by-fame analysis of the video footage – with accuracy rates that were very close to human observation – it was unable to deliver information about the precise number of motorcycles that had actually passed the observation point. “This is because, depending on how fast or slow a single motorcyclist is going, they may show up in three, five or more frames”, says Lin. However, the algorithm was unable to tell whether the motorcycle in the first frame was the same as the motorcycle in the third frame, for instance. Technically, it was unable to count the motorcycles it was observing.

“The advantage of having an algorithm that can track and therefore count individual motorcycles is evident”, continues Siebert: “Letʼs imagine a motorcycle with a tall rider who is wearing a helmet coming towards the camera. As it passes through the first few frames, what the system may not detect is that there is a child without helmet perched on the seat behind the driver. The child becomes visible only as the motorcycle passes the observation point and the camera angle changes. Our previous algorithm would not have been aware that it was looking at the same motorcycle. It would have detected multiple motorcycles with multiple riders and driver/passenger constellations”.

The next step: tracking motorcycles using CNN-based multi-task learning

Fast forward to late 2020, the current publication in IEEE Access and a hands-on solution to Lin and Siebertʼs tracking problem: a multi-task learning (MTL) framework that is based on a convolutional neural network (CNN) and designed to increase both computational efficiency and detection accuracy. Following a three-step approach, Lin and his colleagues fine-tuned a pre-trained RetinaNet algorithm (1) to automatically detect active motorcycles at a frame-based level. Next, (2) they tracked each detected motorcycle through adjacent frames, using both the motion state of the motorcycle and the visual similarity between active motorcycles. In a last step, the system (3) identified the number of riders, their position and whether they were wearing a helmet or not.

One very simple way to track motorcycles through multiple frames is to take the existing frame-by-frame data and add another parameter, i.e. locomotion, explains Hanhe Lin: “Since we know for a fact that the motorcycle in question must be moving, it is possible to calculate its position in the subsequent frame based on the position it was in in the previous frame”. To overcome the related challenge of moving patterns, i.e. motorcycles going in the same direction or in opposite directions, partly obscuring one another, Lin and Siebert turned to common or appearance features. Using a multi-tasking approach, Lin trained the algorithm to detect distinguishing features such as the ridersʼ clothes or what the motorcycle looked like, which boosts identifying and tracking of a single motorcycle even if it moves off-camera for a couple of frames.

H. Lin et al.: Helmet Use Detection of Tracked Motorcycles Using CNN-Based Multi-Task Learning, IEEE Access (2020)

https://www.youtube.com/watch?v=gsJ1hCu5fZc&feature=youtu.be

Improved computational efficiency and detection accuracy: Showing footage from the same observation site, this video shows the CNN-based multi-task learning approach Lin and his colleagues developed to detect and track motorcycles and identify riders and whether they are wearing a helmet or not. The tracking component allows the researchers to track motorcycles going in different directions or partly obscuring one another, also providing improved performance with regard to tracking a single motorcycle even if it moves off-camera briefly.

The results of the study, which is the first study to implement CNN-based tracking of active motorcycles, indicate that the new MTL approach can be used to generate reliable estimates for motorcycles, riders, and position-specific helmet use, demonstrating that artificial intelligence is capable of emulating human observers in terms of delivering accurate helmet use information. “The tracking component of our approach was born out of a very real necessity”, concludes Siebert. “And while this framework will require additional fine-tuning, we are convinced that our approach is worth pursuing”.

And what about data privacy?

Both Lin and Siebert, from their various vantage points, are acutely aware of the issue. “What weʼre doing is not at all about putting people on the spot”, says Lin. In fact, to use the word “tracking” in this context may be misleading in itself. As Lin explains, the algorithm would not be able to tell whether a motorcycle that passes the camera is the same motorcycle it observed five hours ago. Once it has been gone from the image frame for a couple of seconds, the data is discarded: “There is no way for us to come back a few days or even a minute later and go ‘Oh look, hereʼs that guy passing through again’”.

The researchers did store some statistical data for a certain period of time, including information about how many motorcycles passed the observation point, how many riders there were, and how many of them wore helmets. “However, what we do in terms of tracking is limited to a very narrow set of parameters and a very short period of time, and none of it allow us to make inferences about the individuals involved. For instance, we do not track licence plates and we do not store data on individuals who were observed to not wear a helmet”, elaborates Siebert. Both researchers are very clear about their intention:

“Our hope is simply that, given the sheer volume of traffic these days, automated detection of safety-related behaviour such as helmet use can help with developing efficient education and enforcement campaigns and, ultimately, lead to improved road safety for vulnerable road users”. (Dr Felix Siebert, University of Jena, and Dr Hanhe Lin, University of Konstanz)

Whatʼs next for AI-driven road safety?

In fact, Lin and Siebert, in collaboration with researchers based in the UK, Switzerland, and Sweden, have recently submitted a funding proposal for a research project that will see their insights from South-East Asia applied to European contexts – with a special focus on data security and privacy. “European decision-makers are aware that road-related injuries and fatalities pose a major challenge and are looking towards AI for potentially useful interventions”, explains Siebert. Based on this, they are planning to look into ways of bringing the advantages of AI to bear on road safety in Europe while also keeping in mind any ethical, legal and data privacy issues. “From a technical point of view, the main challenge will lie in developing AI applications that conform to the required ethical, regulatory and data security guidelines”, says Lin – a challenge that both researchers look forward to tackling in future.