Why Evidential AI Has to Be Layered

AI is now involved in most video investigations, one way or another, from clarifying footage and spotting a person or vehicle to flagging a moment worth a detective’s time. That’s a good thing. But once an AI system’s output becomes part of the evidence in a criminal case, it stops being judged like a consumer app and starts being judged like a witness: can it be cross-examined, can its process be explained, and can someone show it was wrong?

Courts on both sides of the Atlantic are already answering that question, and a pattern has emerged. It isn’t “AI is too unreliable for evidence.” It’s that a single, opaque model asked to do everything (detect, enhance, identify, decide) fails in predictable ways, while a system built in layers, with a deterministic foundation, a validated learned layer, and a human decision point in between, tends to survive scrutiny. Below are the cases and rules that make that argument for us, so it isn’t just a vendor’s opinion.

Two kinds of AI, and only one belongs near evidence

Kinesense’s own technical team, Chief Technology Officer Dr Mark Sugrue and AI Development lead Dr Michael O’Byrne, draw a distinction internally that’s worth repeating in public. Almost all AI in circulation today falls into one of two camps.

The distinction isn’t academic. In one internal demonstration, the team ran five minutes of CCTV, showing a person in a white hoodie loitering around a vehicle in a car park, through a general-purpose GenAI video-description model. It came back confident, fluent, and wrong: it described a parked car and “no other significant events or movements observed,” missing the loitering entirely. A model trained to sound plausible about the whole internet has no idea what a video investigator actually needs flagged.

That’s not a one-off failure mode, either; it’s the same mechanism behind one of the more widely reported AI controversies of the last few years. In 2020, a generative face-reconstruction tool called PULSE was given a pixelated photograph of Barack Obama and asked to “restore” it. It produced a plausible-looking but entirely fabricated white man’s face, because the model wasn’t recovering lost detail; it was inventing detail that fit its training data, and that training data skewed white. Swap “restore a face” for “enhance CCTV so a jury can see it,” and you have the exact process a Washington court excluded four years later, for exactly this reason.

When the model can’t explain itself, the evidence doesn’t survive

That 2024 Washington ruling is State v. Puloka (King County Superior Court). The defence tried to enter cellphone video that had been run through a consumer AI enhancement tool to sharpen a nightclub shooting. No one disputed the original recording, only the AI-generated version. The court excluded it, finding the tool used opaque methods to reconstruct pixels the camera never actually captured, that its process hadn’t been peer-reviewed or generally accepted in the forensic video community, and that nobody could show it was reliable. It’s widely reported as the first US criminal ruling to squarely address AI-enhanced video, and prosecutors and defence lawyers are already citing it in other cases.

Wisconsin’s State v. Loomis (2016) made a related point about risk-scoring rather than video: a judge partly relied on a proprietary “black box” algorithm, COMPAS, to help decide a sentence. Because the vendor treated its methodology as a trade secret, the defendant couldn’t test how the score was reached. The Wisconsin Supreme Court let the practice continue, but only with a mandatory written warning about the tool’s limitations, and it barred COMPAS from being the sole or determining factor. Even a court willing to allow proprietary AI into a courtroom wouldn’t let it stand alone or unexplained.

When one AI match is treated as an answer, innocent people get arrested

Facial recognition has produced the clearest evidence of what happens when an AI candidate match is treated as an identification rather than a lead. Robert Williams was wrongfully arrested by Detroit Police in 2020, the first publicly confirmed case of its kind in the US, after a facial recognition system flagged him; the case eventually settled after a multi-year ACLU lawsuit. In 2023, Detroit police arrested Porcha Woodruff, eight months pregnant at the time, for a carjacking after a gas-station video was run through facial recognition and a victim then picked her from a photo lineup built from that result, even though the actual suspect hadn’t been visibly pregnant two weeks earlier. She was held for roughly ten hours before the case was dropped for lack of evidence. A federal judge later dismissed her civil claim against the individual officer on narrow legal grounds, while still calling the arrest “troubling for many reasons.” Woodruff was at least the third person wrongfully arrested in Detroit alone on the back of a facial recognition mis-match.

Detroit’s own police chief attributed the failure to “poor investigative work” rather than the technology itself, which is really the point: the technology produced a candidate, and the investigation treated it as a conclusion. Detroit has since changed policy so officers cannot arrest on a facial recognition result alone or build a photo lineup solely from one.

Even a well-built classifier needs a human in the loop

Of course, this doesn’t mean a validated, narrow classifier is infallible; it means its failure modes should be known and disclosed rather than hidden. Kinesense’s own sensitivity testing on person-attribute recognition, for instance, is explicit about where confidence drops: glasses and headwear are hard to call when a subject is far from camera (the team’s own recommendation is to rely on clothing colour instead, which holds up better at a distance); hair length is unreliable when occluded by the angle of a person’s head; and overlapping subjects, two people walking side by side, can mislead the model entirely. The same review flags that most public person-attribute datasets available for training are heavily concentrated in a small number of cities, which is exactly the kind of skew that produces the demographic bias regulators worry about, one reason the team trains instead on more representative, real investigative footage.

Don’t assume “the computer is always right”

No discussion of computer evidence and English law is complete without the Post Office Horizon scandal, not an AI failure, but the cautionary tale that everything above is trying to avoid repeating. Since 1999, English law presumed that a computer system was working correctly unless proven otherwise, a rule that followed the repeal of section 69 of the Police and Criminal Evidence Act 1984. The Post Office relied on that presumption to prosecute more than 900 subpostmasters for theft and false accounting based on unexplained shortfalls reported by its Horizon accounting software. It is now recognised as one of the worst miscarriages of justice in UK legal history, and Parliament is actively debating whether to scrap the presumption altogether, which would shift the burden onto whoever relies on the system to actively demonstrate it’s reliable, rather than leaving the defendant to disprove it.

That’s the direction AI evidence is already heading in practice, and it rewards exactly the kind of system that can show its working from the outset, an unbroken record of what was ingested, what process ran, and what a human did with the result, rather than one that asks a court to simply trust the output.

Where the rules are already going

This isn’t just case law catching up after the fact. The regulatory direction is now explicit:

Agencies buying video AI today are, in effect, buying against a standard that’s already been written down, even where it isn’t yet legally binding in their jurisdiction.

What a layered approach actually buys you: five requirements

Kinesense’s own engineering team distils what they build against into five practical requirements: models trained on real police data, tools that are proven in court, repeatability, reliability, and a full audit trail. Line those up against the cases above and each one turns out to be the direct answer to a specific way things have already gone wrong:

Sources