We let AI answer support tickets for a week. It one-shotted 70 out of 104.

Update timeline

1. monitoring May 15, 2026, 12:00 AM UTC

   If you contacted our customer support recently, you probably talked to Lucie. She is great, but she was on vacation last week and I didn't want to deal with a lot of support tickets by myself. So I thought that maybe it was time to go YOLO-mode and see if our AI features were actually usable, or if we were lying to ourselves. On Friday, while my co-founder Alex was on a plane — he would've never approved this — I turned on this automation rule: Whenever a new ticket came in, AI generated and sent the first reply automatically. To be clear: this was not completely reckless. AI had already been generating draft replies for us on all new tickets for a while. But drafts are one thing. Sending AI replies directly to customers is another. I was a bit scared of this experiment and I honestly didn't know what would happen. What happened: out of 104 tickets we got that week, 70 were one-shotted by AI. By “one-shotted” I mean resolved instantly without human intervention. The customer got an AI reply, it answered the question, and the ticket did not need me or anyone else to step in. I still reviewed all the replies after they were already sent, but other than that it didn't require any effort from me. 70 tickets were resolved with one generated reply instantly. Roughly two thirds of our support load was gone. The tickets were different: technical and non-technical, easy and hard, billing, bug reports, feature requests. The rest — about 30 or so tickets — were handled with an easy follow-up from me, AI-generated more often than not. I was able to focus on the tickets that actually required me. This was all done with our current AI features . I wanted to use this article to show how the setup worked and do a kind of “State of Jitbit AI” thing. Small note: AI changes fast and Jitbit changes with it. We constantly add features, tweak stuff, etc. Most of this article should stay relevant, but some of it could become outdated quickly. Why it worked The truth is AI responses are only going to be as good as the context we provide. It's not entirely magic. The model needs information to compose useful responses. In Jitbit we get that context from several sources: Your Knowledge Base. AI can search it and retrieve relevant stuff. That's good, but not everyone uses KB, it's empty for new customers, and it could be an unmaintained mess like ours. External docs. You can give us a URL to your docs, marketing site, or anything publicly accessible via internet and we index that information for AI to use. Ticket history search. This is the newest addition and maybe the most valuable one. AI can now search your old tickets to see how similar issues were resolved before. External tools / MCPs. AI can call tools you connect to Jitbit. More on that below. We have a bunch of information in KB and our docs are pretty good. Those sources alone were probably enough to resolve a lot of tickets. But I wanted to go deeper. The billing tool I absolutely hate dealing with tickets about billing. “What's the status of my purchase order?” “Why did my subscription expire?” “I've sent a check with a pigeon last month. Did you get it?” My brain dies. Hate it. So I had an idea. We added the ability to connect external tools to the AI stack. This is going to get a tiny bit technical — bear with me. External tools are just HTTP endpoints that Jitbit AI can call in a given format to do something. So I added a tool called ask_billing . It was vibecoded in about 30 minutes. It's a thin HTTP wrapper around Claude Code. Basically, a sub-agent that can deal with billing issues. This tool had instructions to use our payment provider API and look up orders, accounts, subscriptions, etc. I also added the ability to generate quotes and change contact details. I didn't want it to be able to do dangerous stuff like issuing refunds, so it's mostly read-only information. In normal-person terms: Jitbit AI could ask a separate billing agent to look things up instead of hallucinating or making me do it. That worked wonders. It started handling all the billing BS for me. The engineer tool Another tool I added is called ask_jitbit_engineer . It works the same way: thin Claude Code wrapper, but this time with access to our GitHub repo. Our docs are never going to be perfect, but the code always has the most recent answers. The main model in Jitbit can now ask this sub-agent very specific technical questions and get detailed responses. So instead of guessing based only on docs, AI can ask: “Where is this setting stored?” “Who can change ticket statuses and which settings does it depend on?” “How does this API endpoint actually behave?” And then use the answer in the customer reply. Was it perfect? No. Some replies were a bit too “AI support agent”. Some were not how I would have phrased them. Some tickets still needed a follow-up. I would not let it issue refunds, delete accounts, cancel subscriptions, or do anything irreversible. But for first replies and boring-but-answerable tickets, it was already good enough. Actually, better than good enough. It removed most of the repetitive first-response work for a week. What this means for Jitbit customers The built-in AI features are available to Jitbit customers on all plans at no additional cost. That includes AI replies, KB search, external docs indexing, and ticket history search. The external-tool setup is more advanced, but the mechanism is there too. You can connect Jitbit AI to your own systems and let it look things up instead of guessing. In our case, the billing tool was hacked together in about 30 minutes, and it immediately made support less annoying. That setup removed most of the repetitive first-response work for us for a week, and it's only going to get better as we improve Jitbit and the models improve too. I know you're tired of hearing about AI everywhere. Sorry, me too. Just wanted to let you know where we currently stand.