AI in customer service: triage, replies and the human

AI in customer service pays off sooner when it works for the team first and only later talks to the customer. Before putting a bot to answer on its own in chat or on WhatsApp, put an agent to classify each new ticket by category, priority and sentiment, summarize the account history and draft the reply that the support rep reviews and sends. These tasks use the knowledge base and the history the company already has, and a mistake in them costs an internal correction without reaching the customer. Automated replies come later, backed by a reviewed knowledge base and a table that says in which situations the agent hands the conversation to a person, and support and CS leadership owns that table.

At Iasbeck&Co we implement CRM and AI agents in B2B operations. In the workshop I ran in early October, the example that drew the most questions was a triage agent running inside the support automation workflow. The uses in sales and marketing are in AI in revenue operations; here I stay with customer service and post-sale, writing for the leaders of those areas, for the executives who approve the budget and for the HR team that will organize training.

Where should you start with AI in customer service?

Most companies arrive asking for an AI chatbot that answers customers with no queue. I propose starting with the tasks the rep does before replying, because there the supervisor sees the agent’s mistake before the customer does. In four to six weeks, leadership finds out whether the ticket categories make sense, where the knowledge base has gaps and how much the team trusts what the agent writes, and decides with data which topic gets automated replies first.

The study Generative AI at Work, NBER (2023), by Erik Brynjolfsson, Danielle Li and Lindsey Raymond, helps make the case to the committee. The authors followed 5,179 customer support agents who started using an AI assistant that suggested replies during the conversation. Issues resolved per hour rose 14% on average and 34% among novice and low-skilled workers, with minimal impact on the most experienced, and customer sentiment improved. For a support leader, the gap between groups matters more than the average: the assistant carries the way the best reps work to the people who just joined, which changes the math of onboarding.

Layer	What the agent does	Who sees the result first	Cost of a mistake
Triage	Records category, priority, sentiment and queue	Supervisor and rep	Ticket in the wrong queue, fixed by the analyst
History summary	Gathers past tickets, plan, contacts and promises	Rep	The rep checks the history
Draft reply	Writes the reply from the knowledge base and the company’s tone	Rep, before sending	The rep edits or discards it
Automated reply	Answers the customer with no human review	Customer	Wrong answer on behalf of the company

Triage: category, priority and sentiment

Triage runs on every ticket and fills fields that someone fills by hand today or leaves blank. The agent needs closed options: categories the team recognizes, a priority scale that combines customer impact and account type, and a short sentiment scale, such as positive, neutral, frustrated and critical. With the category filled in, the automation workflow routes the ticket to technical support, billing or sales.

History summary and draft reply

A rep who opens a ticket from a B2B customer usually opens four other screens to understand who is on the other side. The agent reads the tickets from recent months, the contracted plan, the renewal date and what the salesperson promised, and returns a paragraph before the first reply, pointing out what is missing from the record. For the draft, the agent looks up the knowledge base article that covers the case, applies the company’s tone and cites the article it used; the rep adjusts and sends. The share of drafts sent without edits, by category, shows where the knowledge base is good and where the agent is improvising.

How does a triage agent work inside an automation workflow?

In the workshop, I showed the agent connected to a support pipeline with channels linked to the help desk, such as website chat, the company’s WhatsApp and the support email. Each new ticket created in the CRM triggers the agent, which reads the conversation, transcripts of recent calls, other tickets from the same company, the knowledge base and the brand kit. Instructed to act as a support analyst, it returns category, urgency and sentiment, writes a summary and fills in the ticket fields. In the live example, it classified the case as a product issue, urgent and critical, quoting the sentence in which the customer said they could not issue a single invoice.

Before this agent, the workflow assigned the ticket to the owner of the company or contact in the CRM, without reading the content. With triage, assignment follows the topic. Two leadership choices weigh on this design. The first is to require a justification with every classification, because that is what lets a supervisor audit the agent in minutes. The second is to run the agent in shadow mode for the first two weeks, with an analyst classifying in parallel, and only hand routing over to the agent once agreement between the two reaches a level the team accepts.

Knowledge base: the prerequisite for an AI chatbot

The agent answers with the quality of the knowledge base it reads. If the refund policy changed and nobody updated the article, and half of the solutions live in the heads of senior reps, the customer gets the outdated version in a confident tone. Before any automated reply, I ask support leadership to pick the most frequent topics of the last three months and check that each one has a current article, with an owner and a review date.

A knowledge base ready for AI brings together solution articles, written so that a new rep can solve the case by reading only that text, and the policies that limit the reply, such as deadlines, refund conditions and the list of what the agent never answers on its own, like penalty amounts or contract interpretation. The “no article in the knowledge base” instruction in the box above turns each gap into a work item for the knowledge base owner, and that queue becomes the support content plan for the following months.

When should AI hand the conversation to a human?

Gartner predicted in June 2025 that by 2027, half of the organizations that expected to significantly reduce their customer service workforce will abandon those plans. In the same poll, conducted in March 2025 with 163 customer service and support leaders, 95% said they plan to retain human agents. For a support leader, the practical decision is about the moment the conversation reaches a person, with what information and how fast.

I write these rules in a table before configuring any agent that talks to customers, and support leadership signs off on each row. Each rule states how the agent recognizes the situation from data it actually reads and what it does before passing the case on, so that the person who takes over does not start from scratch.

Situation	How the agent recognizes it	Who it goes to	What the agent does before handing off
Upset customer	Frustrated or critical sentiment, second complaint about the same issue	Senior rep	Summarizes the case and stops replying
Churn risk	Mention of cancellation or a competitor, critical tickets close to renewal	CS owner of the account	Alerts CS with the ticket and the renewal date
Billing or legal matter	Charges, invoice disputes, penalties, contract, personal data requests	Finance or legal	Confirms receipt and the response time, without addressing the merits
Strategic account	Segment or tier field in the CRM	Dedicated support	Notifies the account owner and prioritizes the queue
Agent lacks confidence	No article in the knowledge base, out-of-scope question, classification with no supporting sentence	General human queue	Logs the gap for the knowledge base owner
Customer asks for a person	Explicit request	Next available rep	Transfers right away, with the conversation summary

Requests involving personal data need their own attention, because the agent reads and records customer information, the subject of AI and data protection. The low-confidence rule protects the brand the most and is the one teams most often forget to write: the agent needs explicit permission to say it does not know.

AI customer service on WhatsApp

On WhatsApp, customers write the way they talk, send audio and screenshots and expect an answer within minutes. An agent on this channel needs to tell customers they are talking to an automated assistant, recognize when the conversation has moved beyond what it can solve and hand it to a person in the same conversation, with the summary ready, so the customer does not repeat what they already said. Outside business hours, it says when a rep will pick up, without promising a resolution time nobody has confirmed.

AI in customer success: from ticket to renewal

The fields that triage fills also serve portfolio management. Repeated critical tickets on an account, sentiment that worsens over the months and failures that come back are risk signals that CS only sees if they are recorded in a standard way. With triage running, the account owner receives these signals without building a spreadsheet, and the summary before the renewal meeting includes the support history for the period. I cover account health and renewal rituals in B2B post-sale; AI comes in there as the layer that keeps those fields filled.

The effect reaches other areas. For sales, the most frequent categories show what the customer expected and did not get, often because of a promise made during the sale, and requests for higher-tier features become expansion signals routed by the rule agreed in the SLA between marketing, sales and post-sale. For marketing, the topics that generate the most tickets point to the onboarding content that needs to exist.

Which metrics show that AI in customer service is working?

Measure before turning the agent on, compare the same team with the previous weeks and break the numbers down by category, because a gain on simple questions can hide a decline on technical failures.

Metric	What it shows	Watch out when reading
First response time	Whether triage and the queue move faster	The automatic “we received your ticket” notice does not count
First contact resolution	Quality of the summary, the draft and the knowledge base	Cross it with reopen rate so you do not reward false resolution
CSAT	Customer perception	Compare cases with and without automated replies
Escalation rate	Calibration of the handoff rules	A very low rate may mean the agent is holding on to cases
Reopen rate	Tickets closed without a solution	Rises when automated replies close the case too early
Corrected classifications	Triage quality	High correction in one category signals a poorly defined category

Support leadership should read the escalation rate together with the CSAT of escalated cases: if the customer reaches a human already upset after three bot replies, the handoff rule is late. The agent and the automations around it also need an owner who tracks these numbers, the subject of AI agents.

How do you prepare the support team to work with AI?

HR and L&D spot the need for training through signs in the routine: reps copy the agent’s suggestion without reading it, supervisors cannot say what the agent does on its own and part of the team uses a personal AI chat with customer data because the official tool has not arrived. Training teaches the team to review what the agent produces and to recognize the moment to take over the conversation.

Support and CS leadership writes the escalation table, the categories and the metrics. Reps and analysts practice reviewing drafts and correcting classifications on the company’s real tickets, and rehearse receiving an upset customer who comes from the bot. The knowledge base owner learns to write articles the agent can use, and the help desk administrator configures triggers and usage limits. The NBER study suggests paying attention to new hires, who gain the most from the assistant and are the most likely to accept a wrong suggestion without question. To measure, track for 30 to 60 days the share of edited drafts, corrected classifications, escalations made under the right rule and the ramp-up time of people hired in the period.

A roadmap for support leaders to bring AI into the operation

Close the categories and scales. With supervisors and analysts, define categories, priority and sentiment, with one criterion sentence per option, and replace free-text fields with closed options in the help desk.

Review the knowledge base for the most frequent topics. List the 20 topics that generated the most tickets in the last three months and make sure each has a current article, with an owner and a review date.

Turn triage on in shadow mode. For two weeks, the agent and the analyst classify in parallel. Adjust instructions and categories where disagreement concentrates.

Write the escalation table. Start from the table above, add the name of each destination and the time within which the person must take over, and get sign-off from support and CS leadership.

Release summaries and drafts to a pilot group. Measure the share of drafts sent without edits by category and compare the metrics with the previous weeks.

Open automated replies on one topic and one channel. Pick the topic with the most drafts approved without edits and the lowest risk, turn on automated replies only for it, with the handoff rules in force, and expand topic by topic.

What stalls this roadmap is seldom the support tool. It stalls when the team does not know how to review what the agent writes, when nobody owns the knowledge base or when the rules for handing off to a human live in the supervisor’s head. That is the work of the in-company applied AI training, built on the company’s process and tickets, with groups for support, CS, sales and leadership, which delivers each area’s prompt library and the agent architecture, with what each agent reads, what it executes on its own and who approves the rest.

Frequently asked questions

Will an AI chatbot replace the support team?

In most B2B operations, no. The agent takes over triage, summaries and part of the low-risk replies, and the team handles the cases that call for judgment, such as upset customers, cancellation risk and strategic accounts. Gartner predicts that half of the companies that planned to cut their customer service staff because of AI will drop that plan by 2027.

Where do you start with AI in customer service?

Start with triage of new tickets, with category, priority and sentiment as closed options, running in shadow mode next to an analyst for two weeks. Then release the history summary and the draft reply for reps to review, and leave automated replies for when the knowledge base has been reviewed.

What does the knowledge base need before the agent talks to customers?

A current article for each frequent topic, with an owner and a review date, the policies that limit the reply and the list of what the agent never answers on its own. Set the agent to flag when it finds no article instead of improvising.

How do you know whether the AI agent is escalating the right cases?

Track the escalation rate together with the CSAT of cases that reached a human and the reopen rate. Low escalation with high reopening indicates the agent is holding on to cases it should pass on. At first, review a weekly sample of the cases the agent solved on its own.

Who on the support team needs AI training?

Support and CS leaders, who define categories, handoff rules and metrics, the reps and analysts who review drafts and classifications, the knowledge base owner and whoever administers the help desk or the CRM. HR or L&D coordinates the program and tracks behavior metrics in the following weeks.

Can you use AI for customer service on WhatsApp safely?

Yes, with the agent identifying itself as an automated assistant, with handoff to a person in the same conversation and with clear limits on personal data, billing and contracts. The company also needs to give the team the official tool and prohibit the use of personal AI chats with customer data.