17 March 2023

Since the early 2000’s when drones first came on to the scene, they have significantly evolved in both capabilities and specialties. Of course, drones initially came out of the radio controlled hobby aircraft industry as a way to put a digital camera in the air on an easy to fly aircraft and get a perspective for photographs and videos previously unavailable.  Shortly thereafter, it was realized that aerial photogrammetry techniques could be applied to drone technology that allowed the creation of aerial maps and three-dimensional point clouds without the need of a more traditional (and expensive) aircraft and camera.  From there, on went the incremental evolution of the technology to include better and larger aircraft that could carry better cameras and more diverse payloads.  Generally speaking, the aircraft and the payload sensor were separate devices that really didn’t interact with each other but used the same power source and communication control interface.  Even though the drone operator still did most of the flying, aircraft have the ability to fly preprogrammed flight paths and detect and not run into obstacles. 

The next generation of drones will take advantage of full sensor integration, artificial intelligence, and machine learning to create an autonomous aircraft that will be able to: 

1. Define the subject matter that it needs capture.

2. Define the best and most efficient method or route to capture the data.

3. Navigate itself over or around any obstacles and adapt it’s flight path as required to complete its mission. 

This evolution will greatly reduce pilot workload making the system much easier and safer to operate. It will also improve data collection process by ensuring capture completeness and accuracy.

The first entry into this new generation of drone technology is the Leica Geosystems Inc. BLK2FLY Autonomous Flying Laser Scanner.

The BLK2FLY was developed by Leica Geosystems AB in Switzerland and is the culmination of Leica’s extensive experience in high precision and precise positioning sensor technology.  Leica Geosystems has combined optical imaging sensors, three dimensional LiDAR laser scanning, high precision GPS positioning, high frequency radar detection, and internet connectivity sensors and fully integrated them using artificial intelligence and machine learning with the more common elements of a drone such as the flight controller and communication links. 

Let's take a look at the individual components of the BLK2FLY and then see how they are fused together to create an autonomous flying laser scanner.

The first and most prominent sensor on the BLK2FLY is the LiDAR laser scanner (see image above) mounted on the front of the aircraft.  This laser scanner has two internal rotating laser scanners capturing a combined total of 420,000  points per second.  The scanner field of view is 360 degrees around the lateral axis of the aircraft and 270 degrees along the longitudinal axis of the aircraft.  This means that the aircraft is capturing data out to 75 feet (25 meters) in front of, to the sides of, above, below, and slightly behind the aircraft while it’s flying.  This LiDAR data is not only captured and displayed for the operator, it is also used by the aircraft to map it’s environment in real time.

Five onboard digital cameras provide a view to the operator in front of, left, right, above, and below the aircraft.  These cameras allow the operator to see what the aircraft sees and define the area of interest for collection.  During collection, the aircraft takes a full dome panoramic 8mp image using all 5 cameras every 2 seconds. This panoramic imagery is used to colorize the LiDAR data and provide project visualization back at the office.

The BLK2FLY also has a high precision Real Time Kinematic (RTK) GNSS receiver (i) onboard that can compute the aircrafts position anywhere in the world to within about 3cm (1 inch). 

When these three sensors are “fused” together, the result is what Leica Geosystems calls “Grand SLAM”.  Grand is the RTK GNSS positioning and SLAM stands for Simultaneous Localization and Mapping.  In simple terms, the BLK2FLY is very accurately mapping it’s operational environment in three dimensions in real time.

The benefit is that Artificial Intelligence and Machine Learning can use that internal map to define the area of data that needs to be collected, the most efficient way to collect it, and then automatically collect it while navigating through the environment autonomously without intervention of the operator. 

For example, if the aircraft is scanning the roof of a building and detects an obstacle in it’s flight path such as some mechanical equipment, power lines, or change in the roof line, the aircraft will deviate either around or over the obstacle automatically to continue it’s mission.   This autonomy relieves the pilot from the guess work of what might be on the roof that can’t be seen from the ground and still collects all of the desired data.  Since the BLK2FLY is also scanning slightly behind it as it flies, it is collecting data on the back side of any object that it flies over thus eliminating the need for additional passes over the same object.

If an obstacle is beyond the BLK2FLY user defined parameter go navigate around or over, or the BLK2FLY detects an obstacle behind the aircraft that is not part of it’s internal map, the aircraft will stop and the pilot will need take over control of the aircraft.

In addition to obstacle detection / avoidance, there are a lot of additional safety features built into the BLK2FLY such as knowing how much battery is required to return to the launch point or operator.  As appropriate, the BLK2FLY will automatically return when it needs a fresh battery.  The batteries are hot swapable so the system doesn’t completely such down while changing batteries.  If the BLK2FLY was in the middle of collection, it will automatically return and resume collection where it left off.  Prop guards protect the props in the event there is an object too small (< ¼”) for the system to detect.

The entire BLK2FLY system is controlled using an Apple iPad Pro running the Leica Geosystems BLK2FLY Live app.   The BLK2FLY Live app incorporates standard “gesture” controls familiar to all smartphone and tablet users.  While flying in Live View mode, if you swipe to the right or left on the tablet, the aircraft will rotate right or left.  If you “pinch” to zoom in or out on the video feed, the aircraft will fly forward or backward.  If you “double tap” anywhere in the live view, the aircraft will fly in that direction the number of feet defined in the settings.  In the 3D view, if you tap any of the LiDAR points in the point cloud, the BLK2FLY will automatically navigate to that point regardless of where it is in relation to the point.  Finally, for those pilots used to “joystick” controls, the BLK2FLY live app has virtual joysticks on the screen that allow you to use your thumbs to control the aircraft just like traditional joysticks.

To the point however, what is all this cool technology used for? 

The BLK2FLY is part of a laser scanning portfolio of products designed to capture and create “digital twins” of objects and structures.  Digital twins are used in all types of industries including Architecture, Construction, Engineering, Manufacturing, and  many others.  An architectural firm might create a digital twin of a building that needs to be remodeled for example.  It is very likely that the building has changed significantly since it’s initial construction and is no longer properly represented by it’s original blueprints.  Laser scanning can be used to quickly capture the building as it currently exist in 3D creating a digital snapshot or twin version of the actual building.  The scan data can be quickly rendered into new blueprints then modified as required for the project. 

In the case of the Leica Geosystems BLK portfolio, there are currently 5 products.   The BLK360 G1 and G2 are tripod based laser scanners that are manually moved from location to location within a structure to capture the area.  The BLK2GO is a handheld mobile scanner almost identical to the BLK2FLY used to walk around inside and outside a structure for capturing data more quickly but slightly less accurately than the BLK360.  The BLKARC is a robot mounted laser scanner designed to work with a number of robotic carriers to autonomously scan an area where it might be too difficult or dangerous to send a person with the BLK2GO.  Finally, the BLK2FLY gets to the exterior surfaces that are too high, dangerous, difficult, or time consuming to take one of the other products.

If the BLK2FLY is any indication, the future of drones is autonomy.

Watch the BLK2FLY

Chris Favel Interviews Bryan Baker of Leica Geosystems