It’s May 2026 and we’re all currently facing an uncomfortable tension driven by the current exuberance related to AI. Nearly every industry is under pressure to incorporate AI into everyday tasks, workflows, and decisions.
As AI proliferates into every aspect of our personal and professional lives, we must delineate which tasks can be augmented by AI today, how this will evolve in the future, and the role of human accountability.
Over the last 18 months I’ve been a Senior Customer Experience Strategy Manager at BILL (a financial automation platform). I’m specifically in a role that serves as the connective tissue between customers, product, engineering, sales, operations, legal, compliance, and any other team that is part of how our customers learn, onboard, use, get help with, and grow their reliance in our products.
Customer Experience is not just about call centers, front-line support, or handling escalations. It is a holistic vision of how a customer perceives a brand across their entire journey—from pre-purchase evaluation and onboarding, to daily product usage and potential cancellation.
Here are 3 critical areas where AI is impacting CX, and how discounting the human component in each of these could lead to disaster.
1. Contextualizing Voice of Customer (VOC) Insights
AI is highly capable of analyzing the sentiment, tone, and temper of customer support interactions. However, out-of-the-box AI cannot properly categorize the unique taxonomy of how your customers talk about your products and brand.
Furthermore, CX teams must synthesize VoC data from a myriad of sources: sales calls, onboarding flows, unsolicited in-product feedback, and third-party sites like Reddit or G2. Unsolicited feedback, for example, is great for discovering new feature requests, but it is also a place where customers go to aggressively vent. If you feed this raw data into an AI system without the context of what each source is inherently good or bad for, the AI will provide guidance that isn’t impactful or relevant to your business goals.
CX leaders must intervene by isolating and weighing feedback from their Ideal Customer Profile (ICP) to ensure that it isn’t just the loudest voices that dictate the product roadmap, marketing, and sales motions.
2. Grounding Conversational AI in Reality
Support experiences are shifting toward digital assistants and headless platforms that connect via APIs to complete complex customer tasks. While AI does a great job analyzing these conversations, it fundamentally lacks the context of specific product limitations, real-time processes, or dynamic policies.
A major risk emerges when AI generates guidance that isn’t rooted in reality. For example, a chatbot might read an outdated Standard Operating Procedure and tell a customer, “give me more information, and I’ll forward it to my team when they come back in tomorrow morning”. If that workflow doesn’t actually exist, the AI’s hallucination creates a serious break in the customer experience.
Support guidance and orchestration requires a disciplined human measurement and monitoring system to ensure that the expectations dicated by an agentic response align with operational reality.
3. Aligning Content Management with Product Reality
AI is changing the game for content creation, but it struggles to guarantee that new content reflects the actual state of your product.
When a new product update is deployed, an ecosystem of dependent knowledge sources—including internal training, sales enablement materials, support articles, and external newsletters—must be audited for accuracy. If human strategy isn’t driving the review of AI-generated content changes, you will inevitably face widespread breakages and failures across your support and sales ecosystems.
Why Accountability Cannot Be Automated
The hype cycle around AI often obscures the severe consequences of simply removing humans from the loop. Letting AI act autonomously without oversight invites a cascading series of operational failures.
Without context, you risk acting on poorly weighted customer feedback that derails your business goals.
Without orchestration, your automated support channels will fabricate guidance and workflows that will damage your reputation.
Without content strategy, product updates will result in broken, inaccurate information from sales to support interactions.
AI is an incredibly powerful tool for augmentation, but the ultimate responsibility for the customer journey cannot be outsourced to a machine. To successfully navigate this revolution, CX leaders must remain firmly in the loop, ensuring that AI operates within the bounds of current operational reality and forward-looking strategic accountability.
To end on a now infamous quote from a 1979 IBM training manual…
“A computer can never be held accountable, therefore a computer must never make a management decision.”
AI decision-making: Where do businesses draw the line? | IBM
The views and opinions expressed here are my own. The opinions expressed here belong solely to me and do not reflect the views of past or current employers.