Search Publica

Technology

Governments want AI assistants with a human answer at the end

New procurement rules for chatbots in tax offices, benefits agencies and consulates require a named official to review any decision that affects a citizen's rights.

2 min read

A long aisle between two rows of black supercomputer cabinets lit with blue accents under a white ceiling
A long aisle between two rows of black supercomputer cabinets lit with blue accents under a white ceilingPhoto: U.S. Department of Energy from United States / Wikimedia Commons (Public domain)

Public agencies have spent the past two years quietly testing AI assistants on the questions that clog their phone lines: how to renew a residence permit, why a benefit payment was late, which form a small business needs to register for sales tax. The pilots worked well enough that ministries now want to scale them up. What they are writing into the contracts is the more interesting part.

Across several governments, new procurement guidance sets the same condition. An assistant may explain, summarise and draft, but any outcome that changes what a citizen owes, receives or is allowed to do must end with a decision signed by a human official.

What the rules require

The guidance is detailed. Suppliers must log every exchange, flag answers that rely on uncertain sources, and offer a route to a person within two steps of any conversation. Agencies must publish a plain description of what each assistant can and cannot do.

  • No automated refusals of benefits, permits or tax relief
  • A named official accountable for each category of decision
  • Full conversation records retained for at least two years

Officials say the aim is to keep the speed of automation without the legal exposure that has followed earlier attempts at automated decision-making, several of which ended in court.

The cost of the human step

The difficulty is that the human step is where the savings were supposed to come from. A deputy director at one revenue agency estimated that review requirements would absorb about a third of the staff time the assistant frees up. Agencies are now designing triage systems in which routine confirmations pass straight through, while anything adverse waits for review.

“The machine can say yes quickly. Only a person should be allowed to say no.”

That line, from a civil servant who helped draft one country’s guidance, has become something of a slogan among procurement teams. Suppliers are less enthusiastic, pointing out that the rule shifts the bottleneck rather than removing it.

Where the pilots stand

Early evaluations are mixed but encouraging. Call volumes at two pilot agencies fell by around a fifth, and satisfaction scores rose among users who had simple questions. Complaints rose among those with complex cases, who reported being passed between the assistant and staff without a clear owner.

What happens next

Most ministries plan a second round of tenders later this year. Civil liberties groups want the rules put into statute rather than guidance, so that a future government cannot quietly drop the human step. Several parliaments are expected to debate exactly that before the contracts are signed.

Share this story