Mikael Grev, founder and CEO of defence company Avioniq AB and a former combat pilot, explores in two columns for FSN Perspektiv the complex challenges and opportunities that AI faces in military technology. In this second column, Mikael delves into the possibilities of implementing AI in aerial combat.

Having discussed the general challenges of AI in military contexts, let us now dive deeper into how these challenges can concretely manifest, with a focus on aerial combat. Here, the boundary between human and machine becomes particularly tangible, and we must understand why collaboration between the two is critical for success.

To give a concrete example: it is relatively straightforward to create an AI that is extraordinarily effective, almost unbeatable, at conducting aerial combat if the system is allowed to operate as it sees fit and only clear, known rules apply. However, it quickly becomes very complex, and requires extensive domain expertise from pilots, if one is to create an AI that conducts aerial combat together with the operator, where the latter may possess conditions and information that the AI is unaware of.

A simpler analogy is that it is relatively straightforward to create an AI that plays chess exceptionally well, but difficult to create an AI that collaborates with a human to play equally well. The root of both problems lies in boundary-setting, where human and machine must understand each other in order to work together. An AI cannot today, and will probably never be able to, except in certain specific cases, effectively explain why it acts in a particular way, with the possibility for the operator to adjust certain priorities and conditions that were not pre-programmed.

The AI industry's solution to the above is to make more information available for AI training, simply because it is the only option that avoids the difficult boundary-setting work that requires domain experts who are often not available. In some domains this works well, for example sensor and image analysis, which has a natural boundary with the human operator and other systems. For other areas, such as aerial combat, where the situation involves a continuous balancing of different solutions, it becomes considerably more difficult.

To replace a combat pilot, even during certain combat sequences, AI would need to be fed essentially the entire pilot training curriculum (this has been proposed), risk assessments (which are difficult to quantify), and all documents included in the operation (which are constantly changing), in order for the AI to generate a solution that does not require human understanding. For the foreseeable future, the operator-in-the-loop will therefore need to understand and continuously be able to correct what is happening, drawing on their experience and all the contextual information that changes during a mission.

The above describes two of considerably more characteristics of AI that become problematic in certain military contexts. The consequences of ChatGPT generating an incorrect answer are manageable, but for military products we naturally set higher standards. This is especially true in the aviation domain, with its rapid sequences of events that do not allow for manual verification of AI outputs, and where missteps frequently have serious consequences. This does not mean that AI cannot be used in the military aviation domain; it simply sets higher requirements and demands that problems are addressed early in the development process. It is easy to be seduced by the possibilities of AI and to extrapolate them too far.

At Avioniq, we have used AI for aerial combat since 2016, but from the very beginning we have been aware of its limitations in aerial combat and have not built ourselves into the problems described above. Let us address them one at a time, as the solutions are of entirely different character.

To address the problem of AI outputs being difficult to verify, given the vast number of possible correct answers and the complexity of the solution space, we have chosen, for these solution areas, to use exclusively what we call verifiable AI.

In principle, this means we create smaller, simpler AI systems where inputs and outputs are well-defined and directly verifiable through a simulation of the entire sequence of events. These AI systems still require enormous computing power to develop, with billions of simulations per AI, but the structure is optimised for clarity and verifiability.

Rather than creating an AI that provides the operator with a heading or flight path to follow in order to win the engagement, which is a poorly defined problem given the many components involved in "winning the engagement", we instead pose simpler questions with clearer and well-defined answers. For example: "How many g (how sharply) must the aircraft manoeuvre to evade the incoming missile if it is of the type AA-10C?"

The difference is that here it is possible to simulate the exact answer, given that a model of the missile is available, and to verify that the AI delivers the correct answer, which is a simple number between zero and nine. In technical terms, one can perform an exhaustive statistical verification of the results for all combinations of input parameters using deterministic simulations.

Verifiable AI does not deliver the complete answer in the way that an all-encompassing AI does, which is precisely the point, as it also resolves the second problem: the boundary between operator and AI. Rather than having an AI deliver the complete answer to a problem, which the operator finds difficult to understand and therefore to modify or approve, verifiable AI delivers solutions to exactly the questions that the operator poses...