Augmented reality for pilots
Moscow Aviation Institute has completed work on a project to create a multifunctional helmet-mounted indication system. A new development for the industry will improve the interaction between man and machine.
The work was carried out within under the Government of Russian Federation Decree No. 218 of 09.04.2010. Developed algorithms for interaction between a pilot and an aircraft will be used in the future to create fully automatic control systems for aircraft. Course of action for such systems will be largely similar to the algorithms of human behavior.
Moscow Aviation Institute at the exhibition VUZPROMEKSPO
MAI stand gathered a crowd at the exhibition “VUZPROMEKSPO” in December 2017. Our University together with its industrial partners, Russian Aircraft Corporation MiG and Quantum Optical Systems Company, presented a new helmet for a fighter pilot. The multifunctional helmet system of virtual reality is the result of two-years partnership between the university and industrial plants. They did a great preliminary work.
To secure the pilot
The most important task of creating aviation equipment is to improve flight safety and efficiency in the implementation of piloting target tasks. According to the International Civil Aviation Organization (ICAO), because of pilot errors, happen up to 70-80% of all air crashes. Improving the safety of piloting is achieved by using automation tools that guarantee the simplest character of the pilot's actions. The aircraft uses a friendly interface: display techniques that do not provoke the pilot to take the wrong steps, but, on the contrary, correct his actions. For this purpose, in the framework of the joint work of MAI, MiG and the company "Quantum Optical Systems", has been developed a method of constructing a promising system of indication, unified for various tasks. It allows the pilot to assess the spatial position of the aircraft, as well as predict its further movement, which is extremely important in solving complex piloting tasks.
In order better to show the pilot the picture, formed in the calculator and transmitted to the display screen, was created a multifunctional helmet system of augmented reality. It is capable to reproduce a three-dimensional predictive indication, allowing the pilot to simultaneously perceive the cockpit external environment.
Story of a research
For more than 40 years MAI has been undertaking studies in the sphere of interaction between pilot and aircraft. Professor Alexander Efremov, Dean of the Faculty №1 “Aircraft Engineering”, explained that for this project, such a system of information display was developed that allows to predict the trajectory movement of the aircraft for a wide range of piloting tasks. The aviation technology of the new generation is multifunctional: the pilot must solve the entire range of tasks available for the modern aviation complex. This means that at any time of the day and in any meteorological conditions, he must take off and land on the runway. When flying for long distances, the pilot must own the technique of refueling in-flight. In addition, perform a low-altitude flight in the mode of tracking the terrain in order to overcome enemy air defenses and other tasks. All this allows to display a multifunctional helmet system of augmented reality created by MAI, RSK MiG and Quantum Optical Systems Company. In the future it will become a friendly interface that allows to safely and with high accuracy perform all the tasks facing pilots of 4++ and 5 generation aircraft.
Partnership with industry
Nikolai Yakutin, head of R&D Department of MiG, said that similar systems have not been used in his company's machines yet, but the work has been already progressing on the introduction of such helmet systems into modern aviation complexes for their subsequent operation in MiG aircrafts such as MiG-29 and MiG-35.
In parallel with it, new developments are being introduced into test and training complexes, including those that equipped with a stereoscopic visualization system. To create a stereoscopic effect in the simulator's cabin without the use of stereoscopic glasses, specialists of MiG developed and manufactured a glass, the shape of which corresponds to the form of the protector of the pilot's helmet. On such a glass is applied a special coating, allowing to separate the images, formed by a stereoscopic visualization system for the left and right eye.
On the pilot's helmet is also installed helmet optical alignment sight. It includes matrix, the image from which is projected onto a special lens, which allows the pilot to control the basic flight parameters during piloting. MiG Company and MAI specialists developed a predictive indication which displays on the lens additional information, facilitating the piloting process in solving problems that require precise control (refueling in the air, landing on the runway and landing strip of the aircraft carrier, pursuit of air targets, etc.).
The special tracking system allows you to track in real time the position of the pilot in the cockpit of the aerobatic complex and to correct the image projected onto the screen. Moreover, one of the promising developments is the system for tracking the direction of the pilot view, which allows you to follow the movement of the pupil. Such system will let the pilot spend less time on carrying out the tasks of pursuing, capturing and fixing air targets.
Anatoly Smurov, Head of the development department of Quantum Optical Systems, said that his colleagues also selected certain working ranges of the spectrum, made energy calculations, designed and manufactured optical trackers that could track the position of the pupil. Test engineers call the development “breakthrough” not only for MiG fighters, but also for the development of modern Russian aviation. “If you want this is a new stage in the relationship between the human-machine system,” - noted Smurov.
Security is one of the most important criteria of work. “Pilot training is a long and complicated process, requiring large financial investments; we can not endanger the vision of pilots at all”, - said Anatoly Smurov. And the staff of Quantum Optical Systems, and pilots themselves underwent thorough medical examination by the oculist doctors during the tests in the laboratories. For a few years of working with helmet-mounted indication systems, none of the testers’ eyesight has been deteriorated.
Up to the sky!
The exhibition and presentation of the demonstrator is an important but not the final stage of many years of work. Ahead there will be tests, also in flight. Only buttons and functions on the aircraft are about two hundred, and usually no more than 0.2 seconds are taken for decision making. The system, even if it weights several tens of grams, provides an additional load on the pilot, which multiplies with overloads. After dozens of tests in the sky it is necessary to find out how the system helps the pilot and makes his work easier.
