15 December 2022

By Dawn Zoldi

Autonomous navigation is a critical feature for any mission-critical drone. As more drones are being deployed in GPS-denied and communications-denied environments such as mines, tunnels, and critical infrastructure facilities such as nuclear power plants, oil tanks, and buildings, the more critical reliable onboard navigation and mapping becomes.

ModalAI’s latest integration of PMD’S Time of Flight sensor with VOXL enables comprehensive and reliable mapping for indoor autonomous navigation.

Time-of-Flight Sensors Map Indoor Spaces

PMD’s ToF Sensor technology provides high-fidelity depth sensing up to six meters which is ideally suited for indoor mapping. Like LIDAR, or light detection and ranging, the TOF Sensor measures the time it takes for its infrared laser light pulses to be emitted, reflected off an object and then to bounce back. Software then generates a live 3D point cloud representation up to 60 times per second. Unlike large spinning LIDAR units that see a thin slice of the world around it, the TOF sensor acts more like a camera with a wide lens and no moving parts.

The technology has been motivated primarily by the smartphone industry for improving autofocus, image processing, and augmented reality. The same motivation of reducing size, weight, and power (SWAP) makes this technology well aligned with ModalAI’s goal of integrating modern robotic perception technology into ever-shrinking drones.

“The TOF Sensor is our direction for indoor robotic perception," says the company's Head of Autonomy, James Strawson. "Traditionally, you've needed large and dangerous drones to carry heavy lidar units to do the same thing. The sensor is small and lightweight enough to fit into a smartphone. One single TOF module weighs just 3 grams. Smaller and lighter sensors allow for smaller and safer drones.”

ModalAI knows about cellular and microcomputer technology. It's highly experienced co-founder and CEO, Chad Sweet spent the 20 years prior to creating the company at Qualcomm, where he led efforts in computer vision and wireless communications. With 15 patents granted, he was responsible for groundbreaking projects such as the Snapdragon Flight, which is powering NASA’s Ingenuity, the first drone to fly on Mars.

“One of the advantages to us being in robotics and working closely with the chipset industry is that we have access to these unique sensors,” notes Sweet. “You can’t just go on Amazon to buy these.”

Sometimes, GPS Just Doesn’t Cut It

Integrating this sensor with the company’s VOXL autopilots, equipped with Visual Inertial Odometry (VIO) to fly safely and reliably, creates incredibly robust perception for indoor and GPS-denied environments.

GPS interferences from a degraded, disrupted or manipulated signal can cause significant negative impacts, including the disruption of critical equipment functions that result in property damage, serious injuries or even fatalities.

All of this applies to GPS-dependent drones. A loss of GPS signal results in “lost link.” The FAA defines a lost link as the loss of command and control link between the drone and the ground station. This could send the drone into uncontrolled flight and crash into people or objects.

Interference aside, GPS signals also simply cannot travel through some materials. In fact, they cannot penetrate most solid objects, such as buildings, packed soil (ground), concrete, heavy wood or metal. Thus, generally, GPS cannot work indoors, or even around large metal objects, such as bridges. This renders GPS either completely unavailable, or unreliable, for PNT in those locations.

In instances where a drone can’t use GPS to navigate, VIO can help the drone understand where it is relative to its environment. VIO uses visual and inertial data to provide odometry, or how far the drone has traveled. It measures distances in 3D and acts as the building block for many drone functions, including obstacle avoidance, position control, and disturbance rejection.

TOF sensors can complement the mapping done by VIO. The VOXL SDK includes an open source program called voxl-mapper that combines VIO and TOF data to generate a dynamic 3D map of the world on board the VOXL autopilot while in flight. It then uses this map to generate safe, dynamic flight paths through cluttered and tight indoor environments. This can all be controlled through the voxl-portal web GUI hosted on the VOXL autopilot.

While VOXL Flight remains a powerful tool for pilots, ModalAI’s VOXL 2 is its next generation which builds upon the excellent work the company has already done with VIO.

VOXL 2 + TOF Prevents Pilots From Flying Blind

The assembled-in-the-USA VOXL 2 is the world’s smallest flying supercomputer, with four times the computing power of the previous autopilot generation. At only 16 grams, it is smaller than a business card and lighter than an AA battery. VOXL 2 is powered by the latest Qualcomm Flight RB5 5G platform and integrates multiple image sensors to provide best-in-class VIO.

The VOXL 2 integrates the PX4 real-time flight controller, a state-of-the-art CPU, video encoder, GPU and Neural Processing Unit (NPU), with ModalAI’s open VOXL SDK. The SDK comes complete with the autonomous behaviors required to safely and reliably avoid obstacles, including mapping and planning software and Collision Prevention.

For best-in-class navigation, VOXL 2 can now support multiple TOF image sensors allowing customers to design in more sensed directions for better mapping and obstacle avoidance. Developers can outfit their drone with four TOF units to achieve 360 degree FOV, which would enable faster mapping. VOXL 2 also unlocks the full 6m TOF range thanks to increased signal processing power.

“Longer range is beneficial because a drone doesn't need to get close to walls or obstacles in an environment,” explains Sweet. “In a tight or non-traditionally straightforward environment like a mine or unfamiliar territory, such as an indoor facility, a drone doesn't need to creep closely to the edges of the room and risk being damaged by obscure obstacles.”

Developers can now leverage the combined tech of both the TOF sensor and VOXL 2 to ensure their drone solutions can fly in all environments, including when GPS cannot. 

“Any solution that integrates VOXL 2 with the TOF sensor will be the smallest, smartest and safest drones that can fly indoors and in GPS-denied areas.” says Sweet.

VOXL 2 Interview

Learn More

ModalAI accelerates development of smaller, smarter and safer drones and robots with SWAP-optimized Blue UAS Framework autopilots built in the U.S.A. From home and business security to retail and government applications, the company’s highly-integrated AI-powered modules empower a variety of industries to utilize aerial and ground autonomous navigation systems that communicate on 4G and 5G cellular networks.

Based in San Diego, California, ModalAI was formed by former Qualcomm Technologies, Inc. employees in 2018, and builds on their prior research and development in the drone and robotic markets. ModalAI’s VOXL product line helps manufacturers and independent builders get to market quickly and affordably. For more information, visit www.modalai.com