Police Tech Vendors May Have Incentives to Distort AI-Generated Reports
Companies selling both AI report-drafting software and other surveillance technologies to police departments could use that AI to omit or downplay mentions of their own products in police reports.
What happened
The ACLU argues that police technology companies face conflicts of interest when they sell both AI report-drafting systems and other surveillance tools to law enforcement. The ACLU cited a 2019 incident in which Vigilant Solutions, an automatic license plate reader (ALPR) company, allegedly recommended that police omit mentions of ALPR technology from their reports. A training slideshow attributed to Vigilant stated: 'If the use of the LPR equipment can be left out of the report, that is what is recommended.' Vigilant was acquired by Motorola Solutions in 2019. Motorola now sells ALPR systems, body cameras, surveillance systems, and AI-assisted report-drafting software. Axon, which sells the reportedly dominant AI report-drafting product (Draft One), also sells license plate readers, surveillance cameras, body cameras, drones, and Taser devices.
Context
Companies with control over opaque AI systems used to draft police reports have both motive and technical means to shape what content appears, what is omitted, and how police technologies are described in reports—particularly their own products. The ACLU contends this is problematic in a criminal justice context because defendants and courts may not know what details the AI has filtered out or reframed. The ACLU notes the precedent of Volkswagen using computer code to secretly manipulate emissions tests in its favor. However, the ACLU explicitly states it does not believe these companies are currently engaging in such distortion, only that the opacity of AI and structural incentives make it possible.
What's disputed
Whether AI report-drafting companies are currently using their systems to omit or distort mentions of their own surveillance technologies in police reports. The ACLU found evidence of Vigilant recommending omission of ALPR references in a training slideshow; no current evidence of actual AI manipulation by Axon or Motorola is presented in these excerpts.