Kia ora koutou,

For my part in the Marsden project one of the things I will be looking at is Australia as a case study. I will be exploring what debates are being had and by who related to lethal autonomous weapons (LAWS). I have only recently started on the project, and so my research is very preliminary at this stage.

In beginning to research the LAWS space in Australia, however, there is an interesting tidbit which has stood out that I’d like to highlight.

There is a term which appears in Australian planning documents, which I had not seen in the US or activist literature: human starts the loop. This sent me on a divergent little hunt to explore whether or not this term had been used by anyone else.

For those not familiar with the literature, questions surrounding to what degree a human should be involved in the decision-making of LAWS has played a central role in military, political, and activist debates. A number of parties have settled on the concept of meaningful human control as central to their arguments around banning or regulating these technologies. A definition for meaningful human control has yet to be agreed upon.

This debate on meaningful human control is related to the concepts of human in the loop, human on the loop, and human out of the loop – all of which are common ways to talk about the degree of autonomy a particular technology has, and the relationship between people and that technology. In the Australian Defence Force’s Concept for Robotics and Autonomous Systems published in 2020, human in the loop is defined as ‘the system performs some functions independently but requires a human to perform functions that complete the system’s task cycle’, while human on the loop is defined as ‘the system performs all functions autonomously but a human may intervene to stop or modify the outcome before the task is complete’. Human out of the loop is not used in the same document, but as is evident in the term it implies some form of full autonomy.

Human starts the loop is, then, an intriguing term to find popping up in Australian Defence planning documents. I have found eight mentions of the term so far, ranging from June 2018 to November 2020. There are two defence blogs where the term is mentioned off-hand, three pieces by Senior Research Fellow at the Institute for National Strategic Studies, T.X. Hammes, and three Australian defence documents where the term is used. Using data humanism, I have depicted the emergence of the term human starts the loop below:

The first detailed mention is the ADF Concept for Command and Control of the Future Force. In this document, they categorise types of human-machine teaming as full human control, human in the loop, human on the loop, and human starts the loop. Human starts the loop is defined as ‘a human sets operational parameters and initiates the machines operation; the machine requires no further human interaction to complete its task’.

This is the same categorisation used in the later 2020 Concept for Robotic and Autonomous Systems. Going into greater detail, they match this ‘control context’ with the ‘technical context’ to define four categories of robotics and autonomous systems: remote systems, automatic systems, autonomic systems, and autonomous systems. Autonomous systems are in the technical context ‘goal based reasoning’ and in the control context slide across from the latter half of human on the loop and through to human starts the loop. In listing examples of each kind of system, AlphaZero is the only example of an autonomous system. The Roomba automatic vacuum and Tesla autopilot are listed as examples of autonomic, as while they fall under the umbrella of human starts the loop in the control context, in the technical context they fall under task based reasoning, rather than the goal based reasoning required for a system to be considered autonomous. The Navy’s RAS-AI Strategy 2040 merely lists a definition of human starts the loop in its index, it does not use the term in the body of the document.

T.X. Hammes argues that human out of the loop is ‘a bad definition’, given that humans design and build autonomous systems, providing the parameters by which they will operate. Therefore, he claims, human starts the loop is a more useful term, accurately reflecting the relationship between people and autonomous systems.

Where this all becomes interesting is when it is placed in the context of the activist language around meaningful human control. I can foresee an argument being made for meaningful human control existing in the technical context, while brushing the control context aside with the use of the term human starts the loop. This is not to say such an argument would necessarily be successfully convincing, but rather that this is where the focus may be placed, likely taking place under the auspices of trusted autonomy. ‘Trusted autonomous systems’ are a focus area for Defence. Work in this space examines how to make sure defence personnel trust the autonomous systems they will be using, and improve public perceptions of autonomy. This includes making the case that those who operate the technology both trust that technology and understand what it is doing, and that the technology has been designed and will be used in accordance with international law.

As part of my research I will be keeping an eye on the use of ‘human starts the loop’ and its relationship with the strong focus in Australia on trusted autonomous systems. Stay tuned for any developments!

— Sian

Image Credit: The diagram above is from the ADF’s 2020 document: Concept for Robotic and Autonomous Systems