Автономные боевые системы и искусственный интеллект: проблемы контроля над вооружениями

Arms control of AWS is an emerging issue in the field of international arms control in recent years. One of the reasons is that with the rapid development of AWS and extensive militarization of AI technology, more and more potential risks that will eventually post threats on human beings have appeared. Although AWS brings many advantages that may greatly increase military power, risk and challenges at the ethic and humanitarian level and the impact on international security cannot be neglected.
Since 2013, attention of the international community has been attracted to the emergence of AWS and problems of its arms control. In 2014, the parties held dialogues at the United Nations, and the whole dialogue has been conducted within the framework of the Convention on Conventional Weapons (CCW). The informal dialogue evolved into formal negotiations in 2017, involving more than 70 countries and many scientists and non-governmental organizations. Even though there are issues that slow down the process of setting control during the years, more and more countries have realized that the need is urgent, and the are ready to achieve more actual results. From the current development of the technology and possible scenario that might occur in the coming future, it is possible on the technical level and necessary for the international community to push forward the arms control negotiations of autonomous weapons systems. The !Eleven Guiding Principles”1 is one of the remarkable results in recent years, which shows the determination of the international community to push forward the process of AWS arms control.
Since the technology has caught everyone”s attention, the term !Lethal Autonomous Weapon System (LAWS)” has been used in many researches and articles, while in my research I use the term !Autonomous Weapon System.” Depending on how it will serve the purposes, the word lethal deserves more discussion. The AWS does not have to be lethal. Although there are discussions about !killer robots” is in many contexts, the discussion of how to control and regulate is also within the frame of many study. It is very questionable to consider AWS and AI technology as evil and homicidal technology. There is nothing wrong with the technology itself. It is the user and the way to use it that creates the problem. The designers construct the input and process international law and other normative data into the machine. At the same time, at what level should AI be trained before it can be put on the battlefield has also caught attention. These are the new points of the
1 Annex IV, Report of the 2019 session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems, CCW/GGE.1/2019/3, https://undocs.org/en/CCW/GGE.1/2019/3;
4
problem. In order to avoid misleading in my research, I decided to use the term !Autonomous weapon system” instead of !Lethal Autonomous weapon system.”
Development and deployment of AWSs is known as the #the third revolution in the warfare.”$And it will be necessary to establish a proper rule to regulate the military use of AI and the increasing potential appearance of AWSs in the future. While there are many articles that focus on the technical problem and ethical aspects, international concerns, there is a lack of research in the field of why these are problems and what is happening in the process of setting control. What is the prospects of the solution and what can we expect from the future. AWS has the ability to respond to changes in actual circumstances, which automatic weapon systems do not and can only operate based on human instruction entirely. An automatic weapon system is a step on the way to explore autonomy in the weapon system. This study will focus on autonomous weapon systems instead of automatic weapon systems.
The aim of this research is to find out what are the problem and prospects of setting international control over the further development and use of autonomous weapon systems. To achieve this aim, specific tasks have been formulated:
To understand the advantages and disadvantage within the technical background of the AWS;
To analysis what will going to happened without international control; • To indicate the what is the problem and need of international control;
The background of the technology and its trend will provide the answer with the experience from the existing AWS and AI technology and how it serves the military that satisfied countries. However, all coins have two sides. AWS and AI technology have their shortages that can not be overcome in a short period of time, and how these issues were causing the problems in the scope of accountability, humanitarian and ethic level. Despite the fact that problems are arising from the development of the technology, with the extensive range of application and high exception from countries that AWS will become a game-changer in the future, AWS and AI technology will have negative impacts on strategic stability and international security. Therefore it is necessity to control the development and deployment of AWS in the future. Although only ethical motivations alone are not enough to push the arms control of AWS into reality, already there is progress. CCW as a main platform and will continue to be in the future, there were conferences for years and only very few progress have been made. A breakthrough happened in 2019 when participants for the first time,
•
•
5

reached an !Eleven Guiding Principles”, which is a signal that arms control AWS is possible and it could become a good start for the future process.
The literatures are divided into different groups. One group is about the technical issues like the technical problem2 and technical feasibility of !the meaningful human control.3” There are studies of the barriers and obstacles from the development of technical aspects and the timeline of the capabilities and improvements. Moreover, evaluating the values and purpose of the weapon is also included.
The second group points out some different kinds of global concerns as well as in the terms of generally problem and perspective of AWS. The first part4 would more focus on the international approach to the problem. The development and deployment of the AWS is the potential conflict that will create new problems among international environment. Thus, it is important to make an assessment of the current interaction and approach to control. The second5 part of the group put more emphasis on the issues and perspectives. Despite the fact that the autonomous would bring lots of advantages, there are already challenges6 seems to be serious issues and arouse the attention. The concern not only from the use7 of it, but also from the way how the civil8 society response to such technology. The purpose of such weapon will reflect the potential issue9 it may cause.
Another part of this group10 is related to the security study. These literatures usually point out the current situation in military and strategic and security problem11. The global security always
2Toward the Agile and Comprehensive International Governance of AI and Robotics, Wendell Wallach, Gray Marchant, 2019, Proceeding go the IEEE, Vol 107, No 3;
3 Taming Killer Robots: Giving Meaning to the “Meaningful Human Control Standard for Lethal Autonomous Weapon Systems”, Adam Cook 2019, Judge Advocate General’s School, Air University Press ;
4 Debating Autonomous Weapon System, Their Ethics, and Their Regulation Under International Law, Kenneth Anderson, Matthew C. Waxman, 2017, Columbia Public Law Research Paper No.14-553, Oxford University Press;
5 D6. 2 Guidelines on Regulating Robotics, Fiorella Battaglia, Antonio Carnevale, Huma Shah, 2014, RoboLaw ;
6 Deciding on Appropriate Use of Force: Human-machine Interaction in Weapons Systems and Emerging Norms, Hendrik Huelss, 2019, University of Kent, Global policy ;
7 Intra-Mission: Reflections on Autonomy and Trust between Humans and Machines in Combat, Manabrata Guha, 2019, Workshop on Trusted Autonomous Systems in Defense, University of Oxford, UK
8 “People are Averse to Machines Making Moral Decisions”, Yochanan E Bigman, Krut Gray, 2018, In press, Cognition 181
9 The next arms race? A military ethical reflection on the effects of artificial superintelligence on drone warfare and American counterterrorism, Gabriel Boulianne Gobeil, 2015
10 “Killer Robots” and Preventive Arms Control: 2016, The Routledge Handbook of Security Studies, 2nd Edition, pp. 457-468.
11Autonomous Weapon Systems, the Frame Problem and Computer Security, Michal Klicewicz, 2015, Journal of Military Ethics, Volume 14 – Issue(2), Published by Routlege
6

changes with the ongoing situation like the increasing development of AWS is one of the factors. The uncertainty of AWS will have an impact on the strategic stability12. This type of the literature13 normal focuses on the way how technology changes would have an impact on the future warfare14.
The use of AWS represents a new era in warfare more than other innovations had previously. AWS could be another asset in the military toolbox to replace human soldiers and decision makers under certain circumstances. Thus it raises not only concerns and questions, but also expectations and perspectives. It is necessary for us to confront as far in advance as possible, which could help us to prepare for the future. Therefore the literature15 with the prediction and expectation of the AWS gives valuable ideas and recommendations for the possible solution and use of the autonomous weapon system. Besides, it is very interesting to see that some authors16 like to make comparisons between nuclear weapons and AWS. To learn from the previous experience, similarity and difference between them would be useful.
Although from different aspects they provided the opinions that would help to understand the risk and potential of the system more comprehensively separately, the gap between the wish for using the AWS ideally and meet so much difficulty in setting practical control remains. Therefore my study presents what cause and possibility to bridge the gap. Thus this is the academic novelty of this study.
AWS as the frontier technology of modern science and technology is a typical representative of the military application of artificial intelligence. AI accelerates the development of autonomous weapons systems while the core technology of AWS is artificial intelligence. Because of the basic function of the AWS, the development and deployment of the AWS must be based on mature AI technology. As a branch of the artificial intelligence, the AWS and AI are closely connected together. Moreover, in recent years, the significant advancement of AI will cause major innovation
12 Greg Allen and Taniel Chan%“Artificial Intelligence and National Security”, Brief center study, 2017;
13 Algorithmic Bias and the Principle of Distinction: Towards an Audit of Lethal Autonomous Weapons Systems, Sarah
Shoker, 2019, Digitization and Challenge to Democracy Globalization Working Papers;
14 The Future of War: Could Lethal Autonomous Weapons Make Conflict More Ethical? Steven Umbrello, 2019;
15 Regulating a Game Changer: Using a Distributed Approach to Develop an Accountability Framework for Lethal Autonomous Weapon Systems, T Krupiy, 2018, Georgetown Journal of International Law ;
16 Artificial intelligence: A Revolution in Strategic Affairs, Kenneth Payne, 2018, Journal Survival Global Politics and Strategy, Volume 60 – Issue(5) // How viable is international arms control for military artificial intelligence> three lesson from nuclear weapons, Matthijs Maas, 2019 // The impact of artificial intelligence on strategic stability and nuclear risk, Vincent Boulanin, 2019, sipri
7

and change of the war praxis through AWS directed by artificial intelligence.17 During discussions in an expert meeting hold by the International Committee of the Red Cross(ICRC) in 2014, speakers and participants referred to the concept of #fully autonomous weapon systems”$meaning highly sophisticated weapon systems with #artificial intelligence”$ that are programmed to independently determine their actions, make complex decisions and adapt to their environment.18 The application of AI in military aspects and weapons has greatly improved the military efficiency of weapons, and conventional weapons can obtain unprecedented “autonomy” through the application platform of AWS. Thus it is necessary to put AWS and AI together in the study.