Meaningful Human Control of Autonomous Weapon Systems

Authors:

  • Amanda Musco Eklund

Publish date: 2020-03-20

Report number: FOI-R--4928--SE

Pages: 59

Written in: English

Keywords:

  • meaningful human control
  • human judgement
  • autonomous weapon systems
  • LAWS
  • international humanitarian law
  • international human rights law
  • human-machine interaction
  • artificial intelligence
  • United Nations Convention on Certain Conventional Weapons
  • CCW

Abstract

There is widespread agreement amongst states and civil society that the concept "meaningful human control" is useful in discussions on autonomous weapon systems. The concept has been defined differently in the debate, but there are common factors which can be systemised into the following key elements of meaningful human control: context-control, understanding the weapon system, understanding the environment, predictability and reliability, human supervision and the ability to intervene, accountability and ethical considerations. The discussions on autonomous weapon systems mainly focus on international humanitarian law aspects. However, a perspective on international human rights law is necessary when developing the concept of meaningful human control, for it to be useful also when human rights law is the governing legal framework. Analysing which requirements the right to life of the European Convention on Human Rights would place on a concept of meaningful human control reveals additional key elements such as expressing meaningful human control through national regulations, necessity- and proportionality assessments and procedural obligations. Since humanitarian law and human rights law differ regarding what use of force is lawful, an additional key element is the ability to assess what legal framework governs the use of force. There is disagreement on how the precise meaning of the key elements should be interpreted in specific contexts. Terms like "predictability" and "understanding" do not have a definitive meaning and must be interpreted in each specific application. Central actors do not agree on several questions. Can weapon systems that autonomously select and engage targets be allowed? Must operators always be able to cancel an operation? A challenge for the future debate is to precise what meaningful human control means in relation to specific questions like these, to ensure that the concept is rooted in the operational reality and that it does not become contradictory in relation to existing noncontroversial weapon systems. The conclusion of this report is that the concept of meaningful human control is still not sufficiently defined to form the basis of any regulation on autonomous weapon systems. However, the key elements are a useful starting point for continued discussions on what technology and use comply with humanitarian law and human rights law.