AI customer question answering.

Which customer questions AI should answer

AI customer question answering works when the answer already exists in your data: an order’s courier status, a size chart, stock at a store, a published returns window. It fails when the answer needs judgement about a specific customer’s situation. So the first job is not choosing a tool. It is labelling a month of real tickets into three lanes.

How the workflow runs

Inputs

Customer messages from chat, email and web forms; product records and size charts; published policies (returns, delivery, warranty, price match); order and fulfilment data; courier tracking events.

Systems

Shopify or WooCommerce for orders, Cin7 or the POS for store stock, a helpdesk such as Zendesk or Intercom, and a model with retrieval over approved content only.

Steps

Classify the message; verify identity before any order lookup; retrieve the relevant policy or record; draft an answer citing it; route by lane; send or queue; log sources and outcome.

Human review

Support staff approve every draft in the faults and disputes lane, read a daily 5% sample of automatic answers, and own all escalations within the same business day.

Outputs

Sent answers with linked sources, staff drafts, escalation tickets carrying a summary, and a weekly list of questions the content could not answer.

Measures

Recontact within seven days, escalation rate by category, first response time, staff minutes per ticket, and policy errors found in sampling.

Risks

Misstated rights, invented policy, stale stock, order data shown to the wrong person, and messages that try to instruct the model.

The NZ rules the answers must respect

The Consumer Guarantees Act 1993 guarantees that goods supplied to consumers are of acceptable quality (section 6), and section 43 stops retailers contracting out of it except in business transactions. The Fair Trading Act 1986 section 13(i) prohibits false or misleading representations about any guarantee, right or remedy, explicitly including those under the Consumer Guarantees Act. A bot that tells a customer “sale items cannot be returned” when the item is faulty creates exactly that risk. The faults lane exists for this reason.

Section 13 also covers claims about the goods themselves, so answers about materials or performance must come from the product record, never from the model’s general knowledge.

Illustrative example: a Wellington homeware retailer

Illustrative example, synthetic numbers: two stores and a Shopify site receive 1,400 tickets a month. Labelling shows 45% order status, 20% product questions, 15% change-of-mind returns, 10% faults and damage, 10% other. Assumptions: a person spends 4 minutes on an average ticket; after a pilot, 50% of all tickets are answered automatically, 30% are drafted for approval at 1.5 minutes each, and 20% are handled in full at 6 minutes.

• Before: 1,400 tickets x 4 minutes = about 93 staff hours a month.
• After: 420 drafts (10.5 hours) + 280 full tickets (28 hours) + sampling 35 automatic answers at 2 minutes (1.2 hours) = about 40 hours.

The running cost depends on how you buy it. Intercom bills its Fin AI agent at US$0.99 per resolution (Intercom, checked 19 October 2026), so 700 resolutions is about US$693 a month. Zendesk also bills AI agents per automated resolution. A custom build on Claude Haiku 5.5, at US$0.10 per million input tokens and US$0.50 per million output tokens (Anthropic), costs about US$1 a month in model fees for 1,400 conversations at 5,000 input and 400 output tokens each, but you pay for building, hosting and maintaining it. Compare the full cost of each over two years, not the per-answer price.

What stays with a person

Staff decide every refund, repair or replacement for a faulty item, every goodwill gesture, and every reply to a complaint. They also own the content: when a policy changes, a named person updates the source page the model answers from.

Where retail question answering goes wrong

• Counting the wrong thing. Intercom counts an “assumed” resolution when a customer leaves after Fin’s last answer without asking for more. Track whether the same customer contacts you again within seven days.
• Order number as identity. Order numbers appear on receipts and courier labels. Match the order to the signed-in account or a one-time code first.
• Stale stock. Store stock in the answer is only as fresh as the last sync. Show the time of the count, or say “check with the store”.
• Policy drift. The model quotes last season’s returns window from an old page. Retrieve only from a published, versioned policy set.
• Hostile messages. Customers paste text that tries to change the bot’s instructions. Test for it before launch; see prompt injection testing.
• Support as marketing. The Unsolicited Electronic Messages Act 2007 bars unsolicited commercial messages. A support chat is not consent.

Mining old tickets for answer gaps is covered in our lesson on building customer FAQs from support tickets. For the build itself, see our AI customer support systems service, the Zendesk and Intercom guides, and the retail sector overview, which links this to inventory exception reporting.

Written by the Levy Street team in Wellington. Last updated 19 October 2026.