We pointed an LLM at our ticketing system expecting a chatbot. We got something stranger: a tier-one analyst that never sleeps, never escalates the wrong thing, and finally let our humans do interesting work.
On a Tuesday in January, a junior accountant named Renee filed ticket #44912: “Outlook keeps asking for my password.” By the time the human on shift opened the queue at 8:47 a.m., the ticket was closed. The resolution note said the user’s MFA token had drifted, a fresh one had been issued, the offending cached credential had been cleared from her keychain, and she had been emailed a two-line explanation with a screenshot. Renee had already replied: thanks.
Our humans had not touched it. None of us realized for about forty minutes.
The before
Our IT desk was eight people supporting fourteen hundred. The queue had a steady-state size of roughly two hundred open tickets and a long tail of three- and four-day-olds that nobody loved. Median time-to-first-response was thirty-eight minutes during the day and four hours overnight. About 70% of inbound was, by volume, the same nine problems — password resets, MFA drift, VPN config, printer queues, Slack permissions, license assignments, calendar sync, the eternal Outlook profile rebuild, and a remarkable number of “my microphone doesn’t work” tickets that turned out, every time, to be the operating system’s privacy toggle.
What we actually built
Not a chatbot. The thing that has worked, surprisingly well, is an agent that reads incoming tickets, decides whether it can resolve them, and either resolves them end-to-end or hands them to a human with a structured summary and its best guess. The difference is small in description and enormous in practice.
When a ticket comes in, the agent does roughly this:
- Reads the ticket and any prior history from the same user.
- Checks the user’s current state against the obvious telemetry sources — identity provider, MDM, the relevant SaaS audit logs.
- Forms a hypothesis. Writes the hypothesis down, in plain English, in a hidden field on the ticket.
- If the hypothesis matches a class of problem we’ve explicitly authorized it to fix, it runs the fix and tells the user what it did.
- If not, it leaves the hypothesis and a recommended next step on the ticket and routes it to a human.
The agent’s job isn’t to be right. It’s to make the human’s first thirty seconds on the ticket count.
That last point is the part nobody talks about, and it has done as much for our team morale as the automation itself. When a human picks up a ticket now, they don’t start from a one-line user complaint. They start from: “User reports Outlook prompting for password. Last successful auth was 14 hours ago. MFA token last rotated 19 days ago, which is unusual; the user’s cohort rotates every 7. Recommended action: force token rotation, expect resolution. Did not auto-resolve because user has elevated mailbox delegation and our policy requires human confirmation for credential changes on delegated accounts.” The human can confirm or reject in fifteen seconds. They spend the rest of the hour on the genuinely strange tickets.
What we won’t let it touch
We were strict about the kill switches from day one. None of these are technical limitations — the agent could do all of them — but we decided they belonged to humans and we have not regretted that decision.
The deny-list is longer than we expected to need. It is also the only reason senior leadership signed off on letting the agent take action at all. Every quarter we review the list and, in practice, we have added items more often than we have removed them. The agent’s footprint expands by removing entries from a deny-list, not by adding entries to an allow-list. This sounds like the same thing. It is not.
The team
The thing I was most worried about — that the humans would feel demoted, or replaceable, or bored — has not happened. The work that is left is interesting in a way the work before was not. We have built two internal tools in the last quarter that we would never have had time to build before. One of them, an onboarding orchestrator for new hires, has already saved more time than the agent does. The agent gave the team time to be the kind of IT shop they always said they wanted to be: one that builds things, not one that resets passwords.
Renee, for what it’s worth, no longer files tickets about Outlook. She did file one last week about a strange networking issue at a co-working space in Berlin. The agent took one look and routed it to a human with the note: “Out of scope. Recommend a human conversation; user’s location and corporate VPN policy interact in ways I am not confident about.” That ticket took us a real forty minutes to resolve. It was a good forty minutes.