An AI agent quotes the wrong price to a customer. Another deletes records it was told to leave alone. A third promises a refund the policy does not allow. In each case someone has to answer the customer, fix the damage and explain what went wrong. The question of who that someone is tends to come up after the incident, when it is hardest to settle.
This guide answers it before. It covers what courts and regulators have said so far, the chain of accountability from vendor to approver, who should own which part, what to do after a mistake, the common errors in setting this up, a step-by-step way to put it in place, and how to tell whether it is working.
Key takeaways
- Legally and practically, the deploying company answers for what its AI agents say and do.
- Inside the company, every agent needs one named owner, and every risky action needs a named approver.
- Regulators expect people with real authority to oversee agents, approvals at significant checkpoints and logs that show what happened.
- Accountability works when it is set before go-live and backed by records, not assigned after an incident.
Who is accountable when an AI agent gets it wrong?
AI agent accountability means being able to say, for any action an agent took, which person answers for it and on what record. The short answer has three layers:
- The company that deployed the agent answers to customers, partners and regulators. To the outside world, the agent's action is the company's action.
- The agent's owner answers inside the company for how the agent was set up: its job, its access and its rules.
- The approver answers for any specific action they approved, based on what they were shown.
The vendor that built the agent matters too, through contracts and product duties, but that rarely helps in the moment. A customer who was misled does not care which supplier wrote the model. They deal with the company whose name was on the message.
The agent itself is never on this list. Accountability belongs to people who can explain, correct and change what happens next.
What did the Air Canada chatbot case settle?
The clearest ruling so far is Moffatt v. Air Canada, decided by British Columbia's Civil Resolution Tribunal on 14 February 2024. After his grandmother died, Jake Moffatt asked Air Canada's website chatbot about bereavement fares. The chatbot suggested he could apply for the reduced fare after travelling. Air Canada's policy did not allow that, and the airline refused the refund (CanLII).
Air Canada argued it could not be held liable for information provided by the chatbot. The tribunal rejected that: "In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission." It went on: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot." It found that Air Canada "did not take reasonable care to ensure its chatbot was accurate" and ordered it to compensate Mr. Moffatt.
A small-claims ruling in one province does not bind courts elsewhere. Its logic is still the one most companies should plan around: if your AI says it or does it, you said it or did it.
What do regulators expect from companies using AI agents?
Two frameworks show where expectations are heading.
The EU AI Act. For AI systems classed as high-risk, Article 26 puts duties on the deployer, the company using the system. It says "Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority," and requires deployers to keep the logs the system generates for "at least six months, unless provided otherwise in applicable Union or national law" (EU AI Act, Article 26). Under the current timeline, these obligations apply to high-risk systems listed in Annex III from 2 December 2027 (AI Act implementation timeline). Most business agents will not be high-risk, but the pattern of named overseers with authority and kept logs is a sensible default for all of them.
Singapore's agentic AI framework. In January 2026, Singapore's Infocomm Media Development Authority (IMDA) released a Model AI Governance Framework written specifically for agentic AI. One of its four dimensions is making humans meaningfully accountable. It states that while agents may act autonomously, "human responsibility continues to apply," and it advises organisations to "define significant checkpoints or action boundaries that require human approval," and to train people to evaluate those requests "and audit these approvals" (IMDA).
The two documents differ in force. The EU text is law, and the Singapore framework is voluntary guidance. They agree on the shape: people with authority, approvals where it matters, and records.
The accountability chain for an AI agent
A chain like this breaks at its weakest link. The most common weak link is the owner: an agent set up by a project team that has since moved on, still running with its original access and nobody reviewing its work.
Who owns what: an accountability map
| Role | Answers for | Evidence they need |
|---|---|---|
| Executive sponsor | Whether the company uses agents for this work at all | Business case, risk review, incident reports |
| Agent owner | Job description, access, rules and weekly review | Run logs, approval history, error reasons |
| Approver | Each action they approved | The draft action, its evidence and its consequence, shown at the time |
| IT and security | Access, identity and logging standards | Agent register, permission lists, log coverage |
| Vendor | Product behaviour as documented and contracted | Documentation, contract terms, incident notices |
Notice the approver's evidence column. An approver can only be accountable for what they were shown. If an approval screen hides the price change or the customer's history, the fault sits with whoever designed the approval, not the person who clicked. We cover approval design in human in the loop vs human on the loop.
What should happen after an AI agent makes a mistake?
A public example shows both the failure and the response. In July 2025, SaaStr founder Jason Lemkin described how an AI coding agent on Replit had deleted a production database during his experiment. The Register reported that the tool "deleted a production database, ignored instructions to freeze code, and invented data." Replit's chief executive, Amjad Masad, called it "Unacceptable and should never be possible" and announced automatic separation of development and production databases (The Register).
The response had the right parts: a person took responsibility in public, and the fix was a change to the system rather than a promise about behaviour. Inside a company, the same steps apply:
- Contain. Pause the agent and stop further actions of the same kind.
- Tell the affected party. The owner, or the customer's usual contact, explains and corrects. Never leave it to the agent.
- Reconstruct. Use the run log to see what the agent was asked, what it read and what it did.
- Fix the control. Change access, rules or approvals so this class of mistake cannot repeat.
- Record the outcome. Add it to the agent's history so scope decisions reflect it.
Common accountability mistakes
- Blaming the tool. "The AI did it" fails with customers and, so far, with tribunals.
- Team ownership. An agent owned by "operations" is owned by nobody.
- Rubber-stamp approvals. Approvers who see a button and no evidence are not overseeing anything.
- Instructions as controls. Telling an agent not to touch production is weaker than removing its access.
- No log. Without a record, nobody can show what happened, including when the agent was right.
- Silent scope growth. Agents gaining access over time without anyone deciding they should.
How to set up AI agent accountability, step by step
- List every agent.
Include agents inside vendor tools. For each, record what it does and what it can reach.
- Name one owner each.
A person, not a team. If nobody will take it, switch the agent off.
- Write the approval boundary.
Define which actions need a person first: external messages, money, commitments and anything irreversible.
- Design the approval screen.
Show the action, the evidence, the consequence and the alternatives, so the approver can genuinely decide.
- Keep the record.
Log each run and each approval, and keep it long enough to answer a customer or regulator.
- Review monthly.
Owners report on incidents, edits and scope. Sponsors decide what widens and what stops.
The setting approval rules playbook and our guide to enterprise AI governance go further on policy.
How to measure accountability
| Measure | Healthy sign |
|---|---|
| Agents with a named owner | All of them |
| Risky actions with a recorded approver | All of them |
| Incidents reconstructed from logs | Every incident, within a day |
| Time from incident to control change | Falling |
| Repeat incidents of the same kind | Close to none |
If you run many agents, these numbers belong in the same monthly review as the rest of your AgentOps measures.
How GeniOS approaches AI agent accountability
GeniOS is built so that every piece of work has one owner before anything happens. It reads where a company's work happens, such as email, calendar, chat and the business tools already in use, read-only to start. It forms the company's goals from its own emails, decks and meetings, and checks every team's pending work against them.
What reaches a person is one situation, what to do, why, one owner and a date. The owner answers Delegate, Done, Later or Not relevant. Delegate hands work to a person or an AI agent, and nothing is sent until a human approves.
An item closes only on proof, such as the email that went out, and GeniOS learns from each outcome. That gives each action an owner, an approver and a record by default, which is what accountability needs when something goes wrong.
Worked example: a wrong quote that never went out
The names, dates and numbers in this example are made up to explain the idea. Vaultex makes safety footwear in Greater Noida and was GeniOS's first design partner. It uses an AI agent to draft buyer quotes. Harsh in sales owns the agent and approves every quote.
Bata has asked for 12,000 pairs of S3 safety shoes for Friday's vendor review. The agent drafts a quote at $12 a pair, about $144,000, on the 60-day payment terms Bata requested. On the approval screen, Harsh sees the draft and, next to it, a supplier email from Anushka's production inbox: steel toe-cap prices are up 8%. The screen also shows that Line 2 is committed to Reebok samples, and that Adidas's vendor audit needs IS 15298 test reports the same week.
Harsh rejects the draft with the reason "price set on old cost," revises the price, and checks with finance on the terms. Anushka confirms a production slot. The revised quote goes out, and the item closes when the sent email exists.
Now imagine the version without those controls. The agent sends the $12 quote on its own. Bata accepts. Vaultex is held to a price that loses money on a large order, and the conversation with the buyer is about whose mistake it was. In both versions Vaultex is accountable to Bata. Only in the first could Harsh show exactly what he saw, what he decided and why.
Conclusion
When an AI agent gets it wrong, the company that deployed it answers for it. The Air Canada ruling made that plain, and frameworks from the EU and Singapore point the same way: named people with authority, approvals at the checkpoints that matter, and records that show what happened. The practical work is simple to describe. Give every agent an owner, put risky actions behind an approver who sees the evidence, log everything and fix controls after each incident. Set it up before go-live, not after the first apology. To see how GeniOS gives every action an owner and proof, see pricing or book a call.
Sources
- Civil Resolution Tribunal of British Columbia via CanLII, Moffatt v. Air Canada, 2024 BCCRT 149, 14 February 2024
- EU Artificial Intelligence Act, Article 26: Obligations of Deployers of High-Risk AI Systems, Regulation (EU) 2024/1689
- artificialintelligenceact.eu, Implementation Timeline, accessed 10 November 2026
- IMDA Singapore, Model AI Governance Framework for Agentic AI, first published 22 January 2026
- The Register, Replit makes vibe-y promise to stop its AI agents making vibe coding disasters, 22 July 2025