Ensuring Railway Track safety using Computer Vision

In June 2011, the FRA began collecting suicide statistics from US train companies on a systematic basis. From 2012 to 2017, more than 219 people died by suicide on the U.S. rail system, while another 220 people were wounded as a result of train suicide attempts. With 358 instances, the United States train system saw its highest documented number of suicides in 2015. Rail suicides can have long-term, negative emotional consequences on the victim’s family and friends. Emotional stress extends to train employees, emergency personnel, and onlookers in the case of rail suicides. Identifying measures to decrease these accidents can save lives, money, and time and assist families, train personnel, and first responders in avoiding mental anguish. Furthermore, when train suicides occur, they frequently gain media attention, which might lead to copycat suicide attempts.  These figures most certainly understate the number of train suicides. Even if the medical examiner or coroner concludes that the reason for rail mortality is uncertain, it is reported as a trespass death rather than a suicide.

Source: Volpe Graphic

With the rising cases of trespassing and in the unlikely event of being hit by a moving train. It calls for measures that ensure track safety and obstacle-free movement of trains on those safe tracks. 

While a variety of psychological measures like installing blue lights in railway platforms in Tokyo, to calm the person with suicidal provoking have been implemented and are highly successful in reducing suicide attempts. 

 blue lamps at train stations prevented suicides at those locations. And scientists could even show that the suicides fell by as much as 84%

But these measures are effective only in-station scenarios. When we take into concern the trespassing and suicide attempt that happen outside the station in general the attempts made on barren tracks. A need to implement a trespassing detection system or more specifically a system that tags the incoming landscape with person trespassing alerts becomes a high priority task to implement. 

A Solution:

A trespassing detection and alert system can be developed by leveraging the recent breakthroughs in Artificial Intelligence and Computer Vision in specific. We suggest a Computer vision enabled trespassing person detection system coupled with distance proximity detection which infers the presence of a person in close proximity to the track, labelled as the unsafe zone. The inference can be drawn in near-real-time and can then be used to signal an alert system. 



 The major challenge that unveils itself is the unavailability of a surplus rich quality image/ video data of people trespassing railway tracks. That makes the computer vision model training difficult. The different working scenarios and timings pose another challenge in creating an effective data dump catering to images of all-weather scenarios, different daytime landscapes to name a few.  Lack of data leads to an ineffective deep learning model in production. Thus hampering the whole effort made into development.

Synthetic Dataset Generation:

Because there are minimal to no trespassing people image samples collected throughout the operating time, artificially manufactured ones will substantially aid in the development of detecting systems. Developments in General Adversarial Networks (GANs), when employed with core intelligence of AI and computer vision, prove to be useful in generating near-real like mimicked image data with different permutations of weather and worktime landscape conditions to synthesize an adequate number of trainable image data to be pushed further in the trespassing detection system development pipeline. The core intelligence of GANs is utilized to create a synthetic adequate number of samples of people images from a small quantity of real person image segments. Then use computer vision techniques to artificially create different worktime landscape conditions on a sample of railway track images. The person images generated are then placed on a canvas of artificially created railway track landscapes at different positions as per use case training requirements. 

Source: https://www.mdpi.com/1424-8220/19/14/3075

Data Annotation:

 The synthetic data mimicking real-life scenarios can be annotated for the training of computer vision classification and detection models. You are free to use any labelling tool of your choice. To save your effort we suggest an easy to use plug-n-play, with Auto label on the go – to facilitate labelling at 10X speed, image annotation solution called Labellerr

We use our favourite annotation tool to label the images with bounding boxes for the presence of a person and classification questions to highlight proximity closeness to the track and label the proximity as safe or unsafe. 


Novel Computer Vision object detection algorithms like Faster RCNN can be scripted and trained on the labelled images to detect the presence of a trespassing person. ConvNet aided Classification algorithms can be trained to gauge the proximity of closeness and label the person’s presence as Safe or Unsafe. 

The person detection model and the proximity classification model can be ensembled together to draw inferences about trespassing being safe or unsafe. This ensemble architecture can serve as the core engine of the trespassing detection and alert system proposed for railway companies. 


The model can be served as cloud APIs or embedded into hardware as per the need and requirements. A camera positioned at the front of the railway engine can be employed to take images of the landscape that lies ahead of the train. The camera can be equipped with an image stabilization mechanism to guarantee the intake of good quality and less ambiguous images. Inference can be drawn on these images in near real-time by the served already trained model and an alert system can be configured to raise an alarm in cases of unsafe trespass detections.

Way ahead from here:

 A feedback loop may be configured to account for correct and incorrect detections to optimize model performance with time. The data collected in the process can be annotated and used to retrain the model on real images in place of synthetic ones to assure better performance in the field.

Why Should You Prefer AI-Powered Smart Data Preparation in Building AI-enabled track safety systems?

Every computer vision and deep learning model needs labelled training data. To get highly accurate labelled data an AI-based data annotation platform is best preferred over a manual annotation platform.

  1. Manually annotating data can be a tedious task. Manual Data Labeling Challenges:
  2. Managing and maintaining the quality of data labelling
  3. Workforce management 
  4. Keeping a track of the cost incurred
  5. Compliance with data privacy requirements
  6. The task to ensure data security

Read More on: The Daunting Task of Manual Labeling in Retail and CPG and how Automation Helps

Labellerr – The Fastest and Smartest Data Annotation Platform

Labellerr provides you simple, feature-rich, affordable data annotation solution.

Why Choose Labellerr?

  1. Data Labelling at scale is an important concern for an organization since creating labels on large data sets by hand is often too slow and expensive. Labellerr solves this problem with their agile ML-Powered data annotation platform. 
  2. Work Quality and Worker productivity is difficult to track in the case of crowdsourcing and freelance data labelling services. Hence now with Labellerr’s marketplace, you can choose from our hand-picked and most trusted vendors to get data labelling tasks done.
  3. Domain and context capabilities specific to tasks are limited with workers on crowdsourcing platforms, contractors, and freelancers. 

So, if you wish your data annotation task to be automated and error-free then choose Labellerr. 

Benefits of Labellerr’s Data Annotation Platform:

  1. Label data at 10x speed using Labellerr’s ‘Auto Labeling’ feature
  2. Track work quality and worker productivity with a personalized dashboard experience.
  3. Get relieved from the hassle of reviewing each dataset, instead review only the ones having low confidence scores. 

The above is still just an indicative list, the superior benefit of adopting an automated way of labelling dataset is peace of mind and trust. The idea here is simple – let the machine do all the work for you so that your focus can be on your customers!

If you wish to try the power-packed machine learning embedded data annotation platform – Labellerr then just click here and explore.


For your  AI needs

 Labellerr provides pre-trained track safety models and products that can deliver Plug n-Play APIs with basic functionality to reach clients and can be modified on request easily and swiftly to be integrated with your production pipeline.

Visit our website and mention your use case in brief and our customer engineer will contact you and help you prepare the plan and get you running on a trial with us to validate.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *