29 June 2023

Introduction by Hannu Lesonen

CEO, Anarky Labs 

In today’s world, technology has revolutionised the way we live our lives. From online shopping to virtual meetings, technology has become intertwined with our daily routines. Technology is also used to save lives. Saving lives with drones using augmented reality (AR) is why we put this paper together.

Augmented reality and drones are being used by search and rescue teams, firefighters and law enforcement in order to increase the speed and efficiency of life-saving operations.

Imagine a world where first responders are able to locate people faster than ever before – this is not a distant dream, but a reality that is already here. AR and drones are providing invaluable assistance in first response, giving teams an unprecedented edge when it comes to saving lives.

This white paper will explore how augmented reality can be used to help drone pilots assist in search and rescue operations, firefighting and law enforcement, making hazardous tasks faster, safer and more efficient.

From flying in beyond visual line of sight (BVLOS) environments to detecting survivors in hard-to-reach areas, augmented reality and drones have greatly improved the capabilities of search and rescue teams anywhere they have been deployed.

Augmented Reality for UAVs

Augmented reality, (AR), is software that augments the world around us with digital information. AR can be experienced through the use of devices such as smartphones and tablets, but what’s much more useful for a drone pilot is when wearing AR glasses.

Drones equipped with cameras and sensors and using AR glasses as a heads-up display are able to show the pilot various angles and heights, allow them to see maps, telemetry information, safe fly zones and much more. This can all be shown to the pilot through the glasses without blocking the pilots view of the real world.

This is especially useful to first response teams in search and rescue, law enforcement or other authorities:

• Safety is the primary benefit as you can send a drone to search for survivors in hard-to-reach areas, places that may be dangerous, difficult to reach or too dark to see.

• Drones can provide rescuers with a bird's-eye view of the area, allowing them to get an accurate assessment of the terrain and any potential hazards more quickly than on foot.

• Using AR software the locations of survivors or at-risk individuals can be pinpointed on the map and the location transmitted to teams, saving time and resources.

• Using AR as help for piloting the UAVs, they are more protected than they would normally be from collisions, damage, and getting lost.

Benefits for Search & Rescue 

There are a number of potential applications for augmented reality and drones in search and rescue missions. One of the most obvious uses is to provide rescuers with long distance navigation possibilities. This information could also be used to create simulations that could help rescuers plan out their strategies more effectively. AR technology can be used to display real-time data about environmental conditions, such as obstacles that need navigating around, which could give rescuers vital information about their mission before they even depart.

With augmented reality technology, rescuers would be able to experience the environment from the perspective of a drone in real time, making it easier for them to understand where they are heading and what dangers may lie ahead.

This could help them make decisions faster and with greater confidence, improving their chances of success during a mission even in beyond visual line of sight (BVLOS) situations.

Drones themselves can also be used to detect thermal irregularities or other signs of life that may not be visible from the ground, helping rescuers locate victims even faster.

Benefits for Fire Fighters

Firefighters are the unsung heroes of our society, risking their lives each day to protect us from the ravages of fire. Augmented reality (AR) and drones are exciting new technologies that have the potential to revolutionise the way firefighters respond to emergencies.

Drones (UAVs), in particular, have been touted as a game-changer for first responders at the scene of a fire. AR-powered UAVs can help quickly identify key elements of a situation by flying into areas that may not be safe for a firefighter and allow the drone pilot to quickly provide response teams detailed instructions on how to respond. Similarly, drones provide an aerial view that can be used to survey danger zones and assess the best approach for getting people out safely.

Safety through situational awareness. AR glasses could be used to give firefighters more accurate situational awareness when flying their drone into hazardous situations. Using AR software firefighters would be able to see where hazardous areas are located and tag those areas before continuing, giving their team and firefighters on the ground valuable information.This information would allow them to make decisions quickly without putting themselves at risk. Additionally having AR tell you exactly where your drone is at all times means losses and damage to equipment is minimised, a big factor in many first response cases.

Risk Assessment. Drones could prove invaluable for assessing inaccessible environments such as high rise buildings or hard-to-reach locations like underground structures. Drone pilots equipped with an AR heads up display offer unprecedented access to information that can help firefighters make more informed decisions while on the job—allowing them to assess the risk before sending their teams into dangerous environments.

Benefits for Law Enforcement

The use of technology in law enforcement has been a long-standing conversation, as its potential benefits are widely acknowledged. From the advent of body cameras to predictive policing, the use of technology in law enforcement has continued to evolve over time. Now, with the introduction of augmented reality (AR) and drones, law enforcement could be at the cusp of a new era in terms of safety and protecting citizens.

Through AR, police officers can gain access to real-time data that can help them make better decisions in dangerous situations. Drones also provide an additional layer of security by allowing police officers to survey large areas from the sky without putting themselves in danger. These technologies can bring great benefits for crime prevention, surveillance and emergency response.

Other Potential Benefits

Not only do these technologies have the potential to save lives in emergency scenarios, they can also enhance officer safety during routine operations. Augmented reality and drones can help officers quickly identify potential threats or hazards before entering a situation, allowing them to take appropriate action without putting themselves in unnecessary danger.

Thermal cameras help in low light conditions like dawn or dusk. When AR is also part of the equation then it can help even in the darkest environments with constant situational awareness. Additionally, drone surveillance can alert officers of any suspicious activity – such as illegal gatherings or protests – before it escalates into a dangerous situation. From improved situational awareness to faster response times, this technology could make a positive impact on public safety.

Not only do these technologies have the potential to save lives in emergency scenarios, they can also enhance officer safety during routine operations. Augmented reality and drones can help officers quickly identify potential threats or hazards before entering a situation, allowing them to take appropriate action without putting themselves in unnecessary danger.

Training

Another application of AR and drones is in training exercises. This could increase trainee pilots familiarity with the technology as well as their confidence when using it during real-world operations. Imagine not having to leave the safety of your office to be able to fly complicated missions with your drone. You could use the AR glasses Heads-up display to teach yourself how to fly a drone.

Drone Training for Search and Rescue

Augmented reality and drones have the potential to revolutionise how search and rescue training is conducted. The ability to provide simulations that access challenging terrain, fly in BVLOS situations, in difficult weather conditions from the safety of the headquarters or a training ground make ARUAVs an invaluable tool for locating people in need of assistance when a real situation exists.

• No costly crashes and damage to equipment.

• No sending trainees to different countries to learn how to fly drones.

• Multiple tactical search and rescue scenarios can be programmed into the glasses.

• No weather restrictions.

• No flight permit requirements.

• No insurance issues.

Drone Pilot Training for Firefighting

By incorporating augmented reality simulations into drills, firefighters can practice responding to different scenarios in a safe environment without needing to deploy resources. They literally could use AR to fly a simulation of a drone flying into a burning building in their office or on their training grounds feeding back the information they would get in real life through the AR glasses.

• No costly crashes from trainee pilots and damage to equipment.

• No sending trainees to different countries to learn how to fly drones.

• Multiple tactical firefighting scenarios can be programmed into the glasses.

• No weather restrictions.

• No flight permit requirements.

• No insurance issues.

Drone Pilot Training for Law Enforcement

By equipping law enforcement agencies with the right tools and training, augmented reality and drones offer an unprecedented level of safety and efficiency when responding to emergencies. Similarly training simulations of all sorts of first response scenarios, can help ensure that officers are prepared for any situation—ultimately helping protect both citizens and their colleagues alike.

• No costly crashes from trainee pilots and damage to equipment.

• No sending trainees to different countries to learn how to fly drones.

• Multiple tactical law enforcement scenarios can be programmed into the glasses.

• No weather restrictions.

• No flight permit requirements.

• No insurance issues.

The AR Glasses

AR glasses, or Augmented Reality glasses, are wearable devices that overlay digital information onto the user's realworld view. When used with drones, AR glasses can serve as a heads-up display (HUD), providing essential flight information and real-time data directly in the pilot's field of vision.

The glasses can show key details such as altitude, speed, battery status, and navigation waypoints, allowing the pilot to maintain situational awareness without constantly looking down at a separate controller or screen.

By integrating AR glasses with drones, pilots can operate their aircraft more efficiently and safely, as they can keep their focus on the drone and the surrounding environment while accessing critical flight information at a glance.

Meta Quest Pro

Simply put, Meta Quest Pro is your key to unlocking new use cases and shifting the perspective on what mixed reality (MR) is capable of.

• Custom chip to power rich experiences with improved thermals, 12GB of RAM and 256GB of storage. Wi-Fi 6E enabled delivering 1.6Gbps throughput. Average battery life 2.5 hours and easily extendable with a USB power bank.

• Large field-of-view (FOV) and the most immersive of the devices we have tested. Camera pass-through allows for totally opaque, bright videos. Turning that camera pass-through off can help in concentrating only on the video feeds and maps.

• Good for a command centre or team leader applications inside a vehicle.

• Works with Anarky Labs AirHUD for drones.

Hololens 2

For precise, efficient hands-free work, Hololens 2 is an ergonomic, untethered self-contained holographic device with enterprise-ready applications to increase user accuracy and output. 

• Qualcomm Snapdragon 850 compute platform, 4GB of RAM and 64GB of storage. Wi-Fi 5 support. Average battery life 2.5 hours active use and easily extendable with a USB power bank.

• Fully transparent therefore full situational awareness of your surroundings at all times. Police in emergency and accident situations love this. Visor can also be flipped out of the way if needed.

• Works with Anarky Labs AirHUD for drones.

Magic Leap 2

Magic Leap 2 maintains its users’ actual view of their working environment while seamlessly integrating intelligent, behaviorally realistic digital content within their field of view.

• Dimming technology, 16GB of RAM and 256GB of storage. Wi-Fi 6 enabled Max CPU frequency 2.4Ghz. Average battery life 3.5 hours. 1440 x 1760 pixel resolution AR display.

• Magic Leap 2 is easy to set up, scale, and manage. It's an open source project so it gives us easier access to resources. This allows us to develop faster to cater for specific first response needs.

• Bigger field of vision than HoloLens 2 with an adjustable transparency. Dimming is excellent when operating outdoors, the screen of the glasses can be dimmed globally or partially in segments.

• Works with Anarky Labs AirHUD for drones.

Projected Savings

Precise projected savings can vary significantly based on the specific circumstances, such as the location, scale of the search area, availability and accessibility of helicopters, drone technology costs, and the frequency of SAR operations. A thorough cost-benefit analysis that considers these factors would provide a more accurate estimation of the potential savings. That said there are some easy assumptions based on data available.

Equipment Costs = 75%. Helicopters are expensive to operate, with costs varying depending on factors such as the type of helicopter, fuel consumption, maintenance, and insurance. Operational costs for helicopters can range from $500 to $2,000 per flight hour. In contrast, drone operational costs are generally lower, with a range of $50 to $500 per flight hour, depending on the drone model, glasses used and amount of pilots. Similarly other forms of transport such as lifeboats and deploying ATVs mean that a drone as a spotter can save an average of approx 70%-75% per hour on deploying equipment.

Time Efficiency = 40%. Faster detection and rescue using drones can potentially lead to reduced search and rescue operation durations. This could result in cost savings related to the number of personnel hours required for the operation, overtime expenses, and associated logistical support costs.

Reduced Risks = 100%. While it's challenging to assign specific dollar values to risk reduction, minimizing the risks associated with deploying human teams in dangerous environments could result in cost savings related to potential injuries, medical expenses, legal liabilities, and equipment damage. There is no downside to reducing risks.

AirHUD

AirHUD enables us to display both visible and concealed data, such as distance to buildings, airspace classification or restriction zones, around the drone in real time by using AR glasses. The pilots can keep their eyes constantly on the drone, increasing operational efficiency and safety by avoiding crashes. Our solution can help pilots perform missions or train drone pilots using simulations of drones for a variety of use cases.

Law Enforcement. By equipping law enforcement agencies with the right tools and training, augmented reality and drones offer an unprecedented level of safety and efficiency when responding to emergencies.

Search and Rescue. With augmented reality technology, rescuers would be able to experience the environment from the perspective of a drone in real time, making it easier for them to understand where they are heading and what dangers may lie ahead.

Firefighting. Using AR software firefighters would be able to see where hazardous areas are located and tag those areas before continuing, giving their team and firefighters on the ground valuable information.