Background
Trespassing on railroad property continues to be a significant problem.
- 42% of railroad deaths are a result of trespassing in U.S.
- The number of casualties have increased since 2011.
- In 2020, Federal Railroad Administration (FRA) reported 1,091 trespass casualties, with 536 fatalities.
There is an increased interest from federal agencies in the use of AI and
autonomous technologies to help solve this problem. The FRA has been funding research
into use of Uncrewed Aerial Vehicles (UAVs) to monitor railroads for trespassers.
Goal
The goal of RAIILS was to evaluate the effectiveness of AI-based tools for reducing trespassing on rail properties.
- Assisted in development of machine learning techniques for real-time trespasser detection.
- Implemented AI/analytics-based techniques that can help address the trespassing problem.
- Demonstrated feasibility to adapt techniques to an edge-computing framework by executing on mobile robot platforms (aerial and ground-based).
- Tested proposed solution in relevant real-world scenarios (in the field).
Approach
- Michigan Tech is using AI and edge-computing framework to improve response times and reduce bandwidth requirements.
- The prototype system consisted of an integrated camera/CPU module capable of being either pole or drone mounted.
- Michigan Tech has conducted interviews with government and industry stakeholders to understand the needs and challenges posed by trespassing on rail properties.
Methods
- Our team developed automated detection of trespassers at sites of interest, using AI/machine learning detection algorithm taking place in real-time.
- Used on-board computing built-in to a UAV to analyze input imagery with AI/ML algorithm.
- RAIILS system can automatically send email or text messages on detection of potential trespassers to interested parties.
- Developed benchmark system.
- Using Raspberry Pi.
- Darknet neural network architecture for real-time person detection.
- Developed upgraded system for field deployment.
- Using Jetson Nano with dedicated GPU.
- Moved to MobileNet2 model with modern single-shot detection algorithm.
- Tested thermal imagery for trespasser detection as well with Darknet architecture and Robust Principal Component Analysis (RPCA).
Preliminary Results
Phase 1
The project team’s efforts have resulted in being able to demonstrate real-time person
detection at ~1Hz using a Raspberry Pi + 5MP camera module.
The total cost of this package was less than $100, making it a cost-efficient solution
for widespread deployment.
The team has worked on real-time (i.e, 15Hz) person detection using joint thermal + optical data and an NVIDIA development board. Because data is only transmitted when a detection is made, this is system amenable to deployment in conjunction with ubiquitous (albeit low bandwidth) cell networks.
Phase 2
The prototype system demonstrated a novel, low cost approach to trespasser detection from UAV-sensed data. The significance of the project was in illustrating how AI and remote sensing technologies can work in tandem to address practical rail industry problems.
- Able to consistently verify detection of simulated trespassers, with a typical delay of 90 seconds from detection to cell phone alert.
- Thermal UAV imagery at 15 and 20 ft height was shown to help detect trespassers.
(Bottom) Sparse component of RPCA decomposition of video data.
Publications
You can read more about the project, its results, and its outcomes in this technical report submitted to the Department of Transportation.
Vega, E., Lautala, P., Nelson, D.N., Weinstein, C., Brooks, C.N. and Dobson, R.J., 2024. Railroad Artificial Intelligence Intruder Learning System (RAIILS). Report No. DOT/FRA/ORD-24/38). United States. Department of Transportation - Federal Railroad Administration, Office of Research, Development, and Technology. 29 pgs