A pilot has his hands full simply flying his aircraft. So it is a good thing if the aircraft’s computers can automatically recognise what there is on the ground. FOI has been working on learning-based detectors for more than ten years, work that has paved the way for the development of, for example, the detectors fitted in today’s premium cars with the ability to recognise pedestrians in the dark.
In 2014 FOI was assigned the task of producing a system capable of recognising details on the ground in order to support decision-making by Gripen pilots. This is a job involving several stages. The first and relatively simple stage is to feed into the system images of known objects on the ground such as uninteresting vehicles, structures in the terrain that might resemble targets etc. More difficult is teaching the system to recognise things of which there are not enough good pictures, for example enemy tanks. This is done by theoretically calculating what the object in question looks like and then entering this image. Then comes the really difficult part: teaching the system NOT to alert the pilot for objects that may be similar, or very similar, to objects that do require the pilot to be alerted.
“There will always be false alarms but these need to be kept to a minimum, otherwise the pilot will be overloaded. This necessitates enormous amounts of training data to be fed into the system in order to teach the computers when not to alert the pilot,” explains Mikael Karlsson, Senior Scientist at FOI.
It is important to emphasise that it is still the pilot who makes all the decisions in the event of an alert, but by helping the pilot to find areas of interest, the system provides a better basis for decision-making.
Mikael Karlsson also explains that the system is general in that it functions equally well with image-generating sensors for other vehicles such as cars, ships or unmanned aerial vehicles, so there are a number of civil applications here that may also be of interest.
“Obvious applications include search and rescue, for example locating people on the ground in forest areas from a helicopter, or for use at sea when coastal patrols need to scan large areas in order to locate vessels using false identification or boats carrying refugees in the Mediterranean.”
FOI is today a leader in this field of research and, according to Mikael Karlsson, there are several reasons for this.
“For a long time we have had at FOI a cohesive image analysis group including high-level post-doctoral experts. We also have the advantage of collaboration with Autoliv, a leader in the field of vehicle safety systems, resulting in a product now being sold to consumers. This entails stringent requirements since false alarms are unacceptable.
In 2022 FOI is due to deliver a complete solution for the running and maintenance of a system for recognising objects of interest on the ground in an image-generating sensor system for Gripen.