A Swarm of Quadcopters for Searching Operations

A Swarm of Quadcopters for Searching Operations
A team of researchers from Moscow Aviation Institute and other research centers in Russia wrote an article "An Application of Swarm of Quadcopters for Searching Operations" at the Elsevier on breakthrough technology: searching for missing people in the forest using UAVs.

According to the Interior Ministry, more than 120 thousand of people in Russia go missing annually. Such a large number comes not only due to accidents and violence, but also includes people, lost in the forests. Currently the most effective way to find a lost person is a systematic sweeping through the forest by lines in conjunction with generation of loud sounds in order to attract person’s attention. Unfortunately, in the vast majority of cases the sweeping personnel consists of volunteers. Is very difficult to form such search-and-rescue teams in remote and sparsely populated areas. Sweeping the forest with search parties is furthermore a long and resource-intensive process that limits its application, especially in remote areas, vast regions and in difficult weather conditions.

Unfortunately, today it is the only reliable way to find a person. Sometimes rescue services use helicopters, but in the context of forests it is difficult to discover a person under the tree crown. The lost person must be out in the wide-open space and signal in order to be recognized by a helicopter. Nevertheless, the engagement of a helicopter does not guarantee success of the search operation and is resourceintensive. In that regard, it is used only in exceptional cases. Searching for persons lost is an operation that requires a very large number of personnel, patience, skills, as well as sufficient logistics: water, food, clothing for different weather conditions, etc.

In order to remedy this shortcoming it is necessary to find a new solution, based on modern technologies. We propose to use a swarm of unmanned aerial vehicles (UAVs), flying under the trees’ crowns. This approach is similar to how a person solves this problem. It is thus possible to achieve high accuracy in human.

A New Approach to Handling the Problems of Search Operations

To bring the technology of finding persons lost to a new level, while using minimum of resources and eliminating volunteers’ involvement, we propose to develop a new approach to the problem. The project was implemented within the framework of the contest "Odyssey" from the company "Sistema". The aim of the competition is to create a device or technology, capable of detecting the exact coordinates of the missing person’s location within a radius of at least 10 km in the natural environment (forest), regardless of the time of day and weather conditions. Another condition was communication infrastructure absence. The test site of 10 sq. km woodland area in the forest with one person within was formed for the competition. The task was locate the person using proposed devices and technologies within 10 hours. 


Kirill Schukin, Head of MAI Drone School

It is worth noting, that the surveyed area exceeds the one that can be swept by the rescue team of 10 people. The average movement velocity for a person on the road is 4-6 km/h; in case of passing through forest areas and watching closely in order to find the missing persons the actual velocity value may drop to 2-5 km/h. The width of the line cannot exceed the direct line of sight – on average 15-20 meters.

The optimal solution of this problem can be the usage of UAVs. Their engagement is much cheaper in comparison with helicopters or search-and-rescue aircraft flights. UAVs advantages over other robotic solutions are high permeability and optimum speed.

Currently there are two types of drones control – manual and automatic.

While using the UAV in manual mode, it is possible to precisely survey the required area. This type of control does not exclude the human factor, for example, giving the wrong command for a drone can lead to its damage. Therefore, the main difficulty of manual control is the need for the presence of a qualified operator. It is also necessary to take into account the signal transmission range to avoid signal losses.


Mikhail Kolodochka and Alexander Kunashuk, MAI specialists

In automatic mode the UAV receives a command and performs actions without operator’s participation, for example, a flight on route with specified points. To use the UAV with automatic control you must specify the survey area, as well as the intended route. Therefore, it is possible to exclude uncharted zones. Determining self-location relative to the specified points requires a sustaining GPS signal for a UAV. 

Detection of persons with UAV requires the usage of such technical devices as thermal or infrared imager (reticle) and multispectral camera. Their installation allows to detect a person from afar, as well as regardless of the lighting conditions. 

A key drawback of using a reticle is the reflection and backscattering from the environment (dust, raindrops, snow, fog), especially in the near-field zone, when the size of snowflakes or water droplets are comparable to the aperture of the lens. This limits their usage when flying over forests, especially during the heated term. The leaves of the trees, heated by sunlight, will not allow sensors to observe the surface, where stand there person.

Engaging a group of UAVs (swarm), operating under the trees’ crowns, allows to address this weakness. While being similar to rescue team sweeping, this approach allows to achieve high accuracy for human detection.

As described above, the UAV must operate automatically. Therefore, the UAV must be equipped with a Pixhawk 4 flight controller. Before the flight the operator creates a flight task – a straight path with a set of key points, located at a distance of 100 m from each other. UAV moves along the trajectory from point to point, thereby performing the flight task. Located points have a valid area of 3 meters and are not related to the height, as it is impossible to take into account all peculiarities of the terrain. The first version of the prototype is shown in figure 2.



The drone is equipped with:

– 900 MHz telemetry system. It is use for manual control;
– 2.4 GHz digital video transmission system. It is use to transmit a video stream to the relay drone;
– GPS/GLONASS. It’s required to locate the drone's own position in real time;
– T-motor engine, quadcopter’s propulsion system; – depth camera sensor. It’s used to create depth maps and is necessary for the collision avoidance system;
– 1.2 GHz analog video transmission system for 16 R.V. Meshcheryakov et al. / IFAC PapersOnLine 52-25 (2019) 14–18 transmitting a video stream to the operator;
– recognition system (Nvidia Jetson TX2). A neural network module with a trained human recognition system;
– autonomous navigation system with collision avoidance, a flight controller with collision avoidance system algorithm on board;
– ducted fan with a fairing, designed for protecting the propeller, increases the motors thrust by 20%.

UAV’s collision avoidance system that must be fast, accurate and reliable. The obstacle avoidance algorithm must detect even small branches in advance, and the evasion must be rapid. To realize the collision avoidance system a comparative analysis of existing algorithms was conducted.



As the tests were accomplished, Intel Collision Avoidance Library work combined with Intel Realsense D435 stereo camera was considered as the best solutions. Using the camera, a map of the depths of the terrain is formed. The camera shutter has an angle of sight of 85.2° x 58° and a range of up to 10 m. The camera is located on the front of the UAV, and its viewing angle allows to make a map of sufficient depth for rectilinear motion. Measurements are carried out constantly while UAV is moving.

Avoidance of obstacles is realized via the method of potential fields. Its essence is as follows: the UAV is considered as a material point in space with zero potential. The next key point in the flight task has an attractive potential field, and each obstacle creates a repulsive potential field (its force is inversely proportional to the distance to it). Thus, the UAV moves in the resulting direction.

Swarm Control of Quadcopters 



Mission accomplishment requires to control more than one UAV. Waypoints for the quadcopter are set in the head unit flight controller. At this stage tests are conducted via simulation. It is possible to use a centralized and decentralized control system of the quadcropter . Decentralized swarm management is necessary for different tasks. Quadcopters do not depend on each other and each of them performs its task. In this paper we consider a centralized control system with a "leader" quadcopter (main drone). The "leader" UAV is the central multicopter in the "chain". This UAV moves ahead of the swarm.

We plan to transmit information about the location of the "leader" UAV to the rest ones in the swarm in real time via MAVLink ports. The GPS position of the main drone is adjusted using the known offset and sent to other UAVs as a series of dynamic waypoints. All UAVs will follow the "leader" at given offset distances. In case of failure or malfunction of "leader" quadcopter, the closest drone becomes the main one.

Human Detection Algorithm 

UOpen data and own data, collected in the forest, were used as a training dataset. Today, analyzing global researches in the sphere of development of detection algorithms, we can divide them into two types:

classical detection algorithms, that are not connected with deep learning. Such algorithms include, for example, human detection, based on histograms of directional gradients; on the movement of the object (background analysis) etc.
human detection, based on convolutional neural networks. These algorithms include two-stage (Faster RCNN, Mask RCNN), one-stage (YOLO, SSD, RetinaNet) and cascade (MTCNN) ones. 



Classical detection algorithms work quickly (even without graphics cards), but the quality of detection is not very high, so it is not applicable in search operations. Therefore, convolutional neural network (based on one-stage YOLO algorithms) is used on the UAV as the detection system. Via neural networks a FPV-camera on a gyro-stabilized platform recognizes images of a person [Dwivedi 2019]. Neural network learning is conducted via labeled dataset using the algorithm of error back propagation. From this dataset clusters for the recognition of a person are being picked out, this allows not to congest the computer by other elements. Examples of recognition are shown in figure.

The detection of humans is carried out using the confidence interval: if the accuracy of the recognition reaches the threshold value, the neural network regards the person ti be detected and found. The UAV signals to the operator about the found human, when a person is detected. The operator verifies the data and then makes a decision to continue or to abort the search operation.

Conclusions 

Finding a person is a laborious and energy-intensive task. The importance of search-and-rescue missions is high and they are acute and urgent not only in Russia, but in the whole world. The proposed solution has a significant advantage over standard ones (engaging a helicopter or a chain of people). We have already implemented our approach to detect humans and conducted the collisions avoidance tests. Future tasks are performing video processing on board of UAV, increasing the autonomy of the drones to avoid engaging the second operator, as well as testing the ability of using a radio relay UAV for enabling communication and local navigation.

Meshcheryakov R.V., Trefilov P.M., Chekhov A.V., Diane S.A.K, Rusakov K.D.* Lesiv E.A., Kolodochka M.A., Shchukin K.O.,** Novoselskiy A. K.,*** Goncharova E.,**** *V. A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences Russia . **Moscow Aviation Institute (National Research University) *** Cybernetics Institute of Russian Technological University (MIREA) ****Bauman Moscow State Technical University.

Ask any question