iStock/sommersby Cheng Lin and AJung Moon
Lethal autonomous weapon systems (LAWS, also known as killer robots) are a real and emerging technology that have the potential to radically transform warfare. Because of the myriad of moral, legal, privacy, and security risks the technology introduces, many scholars and advocates have called for a ban on the development, production, and use of fully autonomous weapons [1], [2].
Lethal autonomous weapon systems have the potential to radically transform warfare.
However, banning LAWS internationally is not as simple as signing a treaty. In order for a ban to be effective, there should be a means of accountability and control to ensure that the ban is respected by its signatories [3]. Export control is a regulatory mechanism that controls the distribution of certain goods across a country’s borders and has traditionally been used to track and control the transfer of high-risk commodities, such as weapons of war. The fact that the critical functions of LAWS are largely dependent on software (an “intangible”) poses a number of interesting challenges: Enforcing the export control of LAWS is more difficult than enforcing the export control of weapons whose components are “tangible,” because software can be assembled by both state and nonstate actors and can be easily distributed, implemented, or changed remotely and after the point of sale.
Given the challenges of applying export control to LAWS, the International Panel on the Regulation of Autonomous Weapons (iPRAW) has suggested the use of norm-setting instruments to either complement or replace traditional LAWS regulation [4]. In our recent work [5], we sought to explore this suggestion by addressing the following question: Can open source licenses help regulate the development of lethal autonomous weapons?
We believe that open source technology will underpin future advances in lethal autonomous weapons.
Given the prevalence of open source within AI and the increasing military interest in nonstate technology, we believe that open source technology will underpin future advances in lethal autonomous weapons. In light of this, and the need for nontraditional instruments to regulate LAWS, we analyzed whether an open source license—a document that accompanies all open source projects—prohibiting the use of its source code in the development or use of LAWS could realistically hinder the assembly of a LAWS. The results of our preliminary analysis suggests that while norm-setting instruments such as open source licenses demonstrate theoretical promise in regulating LAWS, their practical success faces some key limiting factors.
Open Source Activism and LAWS
We first conducted a survey of activism within the open source community. We found two broad categories of open source activism. The first consists of smaller-scale initiatives whose scopes are limited to a single open source project, usually involving a contributor modifying their project’s license to restrict who can use their source code, or taking their source code off of the internet [6]–[7][8][9].
The second category involves a newer, larger-scale movement that has developed in open source, where activists draft repurposable open source licenses that allow any developer to limit how their work is used. We collectively refer to these licenses as “ethics-driven licenses.” An ethics-driven license is usually based on an already-popular open source license, with an added stipulation that the software shall not be used for various activities, ranging from the violation of human rights, to environmental degradation, to the violation of labor laws. The goal of these documents is not to restrict a particular project from being used by an organization the authors deem unethical, but rather to give all developers the agency to prevent their work from being used in ways they disagree with. Because the anti-LAWS open source licensing model we study in our paper is an example of an ethics-driven license, the reception of existing ethics-driven licenses is a useful signal for the potential of a license that prohibits a software’s use in LAWS. The three ethics-driven licenses that have received the most attention, and which we look at in detail in our paper, are the Do No Harm License (released November 2017), the Anti-996 License (released April 2019), and the Hippocratic License (released September 2019) [10].
Community cohesion the most difficult success criterion to satisfy for ethics-driven licenses.
Overall, the open source community’s response to the licenses has been poor: The Do No Harm License never reached a stable version, the Anti-996 License only received adoption by small open source projects, and the Hippocratic License only has around 20 projects listed under it [11]–[12][13]. The weak reception of these political licenses is not surprising, however. As Coleman observes, the open source community consistently embraces the “political agnosticism” of their community [14]. In fact, a handful of core open source community members have spoken out against the small- and large-scale activism we have discussed above, arguing that they are a threat to the open source movement [15]–[16][17].
Three Criteria for Success
We found three issues common to ethics-driven licenses. First, many of the licenses are either vaguely worded or have too broad a scope to be meaningful. Second, the legal enforceability of open source licenses is uncertain, as there has yet to be a judicial ruling for a license violation claim. Lastly, the success of ethics-driven licenses requires that other developers respect, adopt, and contribute to projects that use these licenses. However, the open source community has yet to demonstrate such cohesive support for this activism, with groups of open source developers who support the initiative conflicting with those who believe ethics-driven licenses are a threat to the political agnosticism of open source.
These issues also allow us to articulate three success criteria for an ethics-driven license that support a ban on LAWS: 1) clear and specific language developed by legal professionals; 2) confirmation from a credible source of the license’s legal enforceability; and 3) the open source community’s cohesive support of the license. While the first two criteria—language clarity and legal enforceability—are likely within reach, obtaining support from the overall open source community for such a license is still uncertain. This makes community cohesion the most difficult success criterion to satisfy and the ideal focus for future discussions on ethics-driven licenses.
In summary, we explored the potential of open source licenses to help regulate the development of LAWS. While the open source community presently lacks enough cohesion for such an ethics-driven license to have practical promise, the ethics-driven license movement is still developing. The question of community cohesion may be revisited at a later date, when there are more ethics-driven licenses, more discourse surrounding them, more time for projects to adopt them, and perhaps even an ethics-driven license focused on LAWS. At the same time, present activism on the LAWS debate has mostly been focused on generating traditional forms of weapons regulation. Given the unique, intangible nature of LAWS, we should reflect on whether these efforts bring about a solution with practical accountability and control mechanisms, and if not, how we might evaluate and implement new solutions previously unexplored by export control communities.
JOIN SSIT