The human-in-the-loop myth: Why human-in-the-loop is not a control by itself

Abstract

Human-in-the-loop (HITL) mechanisms are commonly presented as a primary safeguard for managing risk in artificial intelligence (AI) and automated decision systems deployed in organisational and workplace settings. The presence of human review is often assumed to ensure accountability, explainability, and safety. This paper argues that HITL is not a control by itself, but a design choice whose effectiveness depends on governance, authority, incentives, and cognitive conditions. In practice, human oversight frequently fails to meaningfully reduce risk, particularly in high-volume, high-impact, or time-sensitive environments. The paper examines the structural and organisational factors that undermine effective human review, including automation bias, information asymmetry, scale mismatch, and accountability diffusion. It concludes by outlining the governance conditions required for human involvement to function as an effective risk control in AI-enabled systems. The paper reframes the role of humans from downstream approvers of automated outputs to upstream decision makers responsible for governing where, how, and under what conditions automation should be applied. This article is also included in The Business & Management Collection which can be accessed at https://hstalks.com/business/.