Moving Beyond AI Experimentation: What Separates Success from Stagnation
The shift from AI pilot projects to operational deployment is exposing a fundamental divide in how companies approach the technology. While many organizations remain trapped in proof-of-concept cycles, a smaller group has figured out how to embed AI into daily workflows without creating new layers of complexity.
This distinction became clear during a recent panel discussion featuring leaders from Zoom, PhysicsX, Portia AI, and Sellmyride. Their experiences reveal that successful AI adoption isn't about having the most sophisticated tools—it's about solving specific problems and understanding the cascading effects of automation.
Monique Koster, who leads small business operations for EMEA at Zoom, points to a counterintuitive pattern: "The companies that succeed start by identifying a real problem and then let AI become part of the solution. The ones who struggle are the ones still stuck in pilot mode." At Zoom, AI companions prepare meeting context automatically and capture key points without requiring users to learn new interfaces or change their behavior.
This approach contrasts sharply with organizations that bolt AI onto existing systems as an afterthought. The difference shows up in adoption rates and actual usage patterns, not just in executive presentations about AI strategy.
The Bottleneck Migration Problem
Emma Burrows, CTO at Portia AI, identifies a critical failure mode that many companies miss: solving one bottleneck only to create another downstream. "Teams need to identify a 'crunch point' in workflows and craft the right tool around it," she explains. "But the failure lies in not understanding where that bottleneck will migrate once you solve the immediate problem."
This forward-thinking approach requires mapping out workflow dependencies before deploying AI solutions. A tool that automates data entry might simply shift the bottleneck to data validation or reporting. Without anticipating these secondary effects, companies end up playing whack-a-mole with productivity constraints.
The distinction matters because it separates short-term experimentation from sustainable AI strategy. Burrows notes that companies with longevity in their AI implementations think in terms of evolving problems, not static solutions. They ask what challenges will emerge in a month or a quarter, not just what hurts today.
This perspective is particularly relevant for scaling companies where processes and pain points shift rapidly. An AI solution that works for a 50-person team may create new problems at 200 people, not because the technology fails but because the underlying workflow dynamics have changed.
Data Quality Determines Model Performance
While consumer AI tools like ChatGPT rely on broad internet-scale training data, specialized applications face different constraints. James Mulligan, head of strategy at PhysicsX, describes how his company trains models exclusively on physics simulations rather than general text data.
"If we're going to predict the outcome of a numerical simulation, we need a whole stack of potential simulations to train that model," Mulligan explains. "The more data we have, the better the model becomes." This creates a chicken-and-egg problem: you need simulation data to build the model, but generating that data can itself be a bottleneck.
The challenge extends beyond just quantity. PhysicsX aims to build foundation models that could predict vehicle aerodynamics from design specifications alone, without having seen that specific configuration before. Achieving this level of generalization requires not just more data, but diverse, high-quality examples that cover the relevant problem space.
For companies considering specialized AI applications, this highlights an often-overlooked cost: data generation and curation. The model itself may be the smaller investment compared to building the training dataset, especially in domains where existing data is scarce or proprietary.
Burrows adds that data governance doesn't have to mean heavy-handed policies that slow adoption. "A better approach is to think about the kinds of ways that people are informally sharing best practices within the company, which is more effective than specific top-down guidelines." This suggests that cultural norms around data usage may matter more than formal policies, particularly in the early stages of AI adoption.
Shadow AI as a Leading Indicator
Mulligan describes an unexpected source of insight at PhysicsX: tracking how engineers use AI tools without formal IT approval. "Shadow AI is the strongest signal for us," he says. "All of our engineers are using AI personally and can explain the value of it to their actual job."
Rather than viewing this as a security risk to be eliminated, PhysicsX treats it as market research. Engineers who adopt tools on their own have already validated the use case and can articulate the value to colleagues. They become internal champions who understand both the technology and the specific workflow problems it solves.
The company reinforced this dynamic by providing experimental budgets—$200 allocations that employees could spend on AI subscriptions like Claude or other tools. This approach acknowledges that the people closest to the work often spot opportunities before management does.
For organizations worried about uncontrolled AI usage, this suggests a middle path: rather than blocking shadow AI entirely, create structured ways for employees to experiment and share what they learn. The alternative—waiting for top-down AI initiatives—often means missing the most practical applications.
Harriet Allardyce, who manages client relationships at Sellmyride, describes using AI to fact-check her own perceptions of sales calls. "If I feel that I've had a positive client call, I will fact-check it with AI that doesn't have bias," she explains. "It's a fantastic training tool and it can help you more accurately forecast relationships."
This application addresses a common problem in customer-facing roles: the gap between how a conversation felt and what actually happened. By analyzing call transcripts without emotional investment, AI can surface warning signs that might otherwise be missed until a client relationship deteriorates.
Making the Business Case Stick
Proving AI value to leadership requires connecting technology capabilities to specific business pain points. Allardyce describes how Sellmyride's distributed team across the US and UK was losing work due to fragmented systems and time zone challenges. "When I first joined, we didn't have a centralized CRM and everybody was using AI here and there which wasn't integrated."
The solution wasn't just implementing AI—it was choosing tools that could serve multiple teams simultaneously rather than creating new silos. Meeting summaries generated for UK-based staff could inform pitches by US colleagues, backed by data rather than verbal handoffs across time zones.
Koster emphasizes that this connection to pain points is what makes AI valuable internally: "I see in small businesses there's a lot of pain points around time and resource savings. Connecting AI to a pain point in an organisation is what makes the technology so valuable."
This framing shifts the conversation from "should we use AI?" to "which specific problems are we solving?" The latter question forces concrete thinking about metrics, workflows, and outcomes rather than abstract discussions about innovation.
For companies still in pilot mode, this suggests a diagnostic question: can you articulate the specific bottleneck you're addressing and how you'll measure improvement? If not, you're likely experimenting rather than implementing.
The gap between AI experimentation and operational deployment isn't primarily technical. It's about understanding workflow dynamics, anticipating secondary effects, and connecting technology capabilities to business problems that people actually care about solving. Companies that figure this out don't necessarily have better AI—they have better questions about where and how to apply it.
