Railroad Artificial Intelligence Intruder Learning System (RAIILS)

Concept of RAIILS project
 Concept of RAIILS project shown above.
Conceptutal diagram of RAIILS System.
RAIILS System Realization Component Diagram.

Background

Trespassing on railroad property continues to be a significant problem.

  • 42% of railroad deaths are a result of trespassing in U.S.
  • The number of casualties have increased since 2011.
  • In 2020, Federal Railroad Administration (FRA) reported 1,091 trespass casualties, with 536 fatalities.

There is an increased interest from federal agencies in the use of AI and
autonomous technologies to help solve this problem. The FRA has been funding research into use of Uncrewed Aerial Vehicles (UAVs) to monitor railroads for trespassers.

Goal

The goal of RAIILS was to evaluate the effectiveness of AI-based tools for reducing trespassing on rail properties.

  • Assisted in development of machine learning techniques for real-time trespasser detection.
  • Implemented AI/analytics-based techniques that can help address the trespassing problem.
  • Demonstrated feasibility to adapt techniques to an edge-computing framework by executing on mobile robot platforms (aerial and ground-based).
  • Tested proposed solution in relevant real-world scenarios (in the field).
AI Detection of railroad intruders
 AI detection of trespassers from a UAV using the RAIILS technology.

Approach

  • Michigan Tech is using AI and edge-computing framework to improve response times and reduce bandwidth requirements.
  • The prototype system consisted of an integrated camera/CPU module capable of being either pole or drone mounted.
  • Michigan Tech has conducted interviews with government and industry stakeholders to understand the needs and challenges posed by trespassing on rail properties.
Raspberry Pi and camera module
Raspberry Pi and 5MP camera module used for initial prototype.

Methods

  • Our team developed automated detection of trespassers at sites of interest, using AI/machine learning detection algorithm taking place in real-time.
  • Used on-board computing built-in to a UAV to analyze input imagery with AI/ML algorithm.
  • RAIILS system can automatically send email or text messages on detection of potential trespassers to interested parties.
  • Developed benchmark system.
    • Using Raspberry Pi.
    • Darknet neural network architecture for real-time person detection.
  • Developed upgraded system for field deployment.
    • Using Jetson Nano with dedicated GPU.
    • Moved to MobileNet2 model with modern single-shot detection algorithm.
    • Tested thermal imagery for trespasser detection as well with Darknet architecture and Robust Principal Component Analysis (RPCA).

Preliminary Results

Phase 1

The project team’s efforts have resulted in being able to demonstrate real-time person detection at ~1Hz using a Raspberry Pi + 5MP camera module.
The total cost of this package was less than $100, making it a cost-efficient solution for widespread deployment.

The team has worked on real-time (i.e, 15Hz) person detection using joint thermal + optical data and an NVIDIA development board. Because data is only transmitted when a detection is made, this is system amenable to deployment in conjunction with ubiquitous (albeit low bandwidth) cell networks.

Xavier developer board and FLIR Duo Pro camera being used for current experiments.
 Xavier developer board and FLIR Duo Pro camera used for Phase 1 Experiments.

Phase 2

The prototype system demonstrated a novel, low cost approach to trespasser detection from UAV-sensed data. The significance of the project was in illustrating how AI and remote sensing technologies can work in tandem to address practical rail industry problems.

  • Able to consistently verify detection of simulated trespassers, with a typical delay of 90 seconds from detection to cell phone alert.
  • Thermal UAV imagery at 15 and 20 ft height was shown to help detect trespassers.
Group of people with a drone on a railroad
Field testing with UAV.

 

Thermal imaging of people on a railroad
Simulated trespassers captured using thermal camera on UAV operating at an altitude of 15ft.
Themeral imaging of 3 people walking.
(Top) Thermal camera input.
(Bottom) Sparse component of RPCA decomposition of video data.