The agile product mindset: 3 practitioners on experimentation, discovery, and prioritization

Product teams everywhere talk about being agile.

They run sprints, hold standups, and maintain backlogs. But three recent keynotes from seasoned product leaders suggest that the real competitive advantage lies somewhere deeper than process, in a mindset built around learning velocity, genuine customer understanding, and disciplined prioritization.

Each speaker approaches the agile product mindset from a different angle, and taken together, their insights form a surprisingly coherent playbook for product leaders who want to move beyond ceremony and into substance.

Experiment velocity as competitive advantage

Avi Schneier, Managing Partner at Agileon, opened his Product Led Summit talk in San Francisco with a reframe that set the tone for everything that followed.

Most people associate agile with process, but he questions: "The truth of the matter is, what's the point of the process? It's to make better products."

Avi's central argument is that customer comprehension alone isn't enough to win. Knowing your customer matters, but the real differentiator is how quickly you can test ideas and learn from them. He calls this experiment velocity, and he's blunt about its importance: "He who experiments most has the chance to win. She who figures out how many iterations we can do wins."

To illustrate the point, Avi reached outside the software world entirely. He described how Pepperidge Farm, maker of Goldfish crackers, typically needed about two years to develop a new snack product. When they adopted agile approaches to product development, they cut that timeline to nine months, a more than 50% improvement in speed to market.

The resulting product, Goldfish Epic Crunch, eventually failed, but the learning it generated led directly to Flavor Blasted Goldfish, which succeeded. The failure was productive precisely because the team could iterate quickly enough to apply what they'd learned.

This example captures something important about Avi's philosophy. He's less interested in whether any single experiment succeeds and more interested in whether the organization can sustain a high rate of experimentation over time. The wins will come, he argues, as long as the learning keeps flowing.

The PACE framework

To help product leaders build this kind of environment, Avi introduced a framework he calls PACE, which stands for:

People and permissions,
Asks as hypotheses,
Cadence and capacity, and
Evidence systems.

The "people" component focuses on psychological safety and lightweight guardrails. Teams need autonomy to experiment, but they also need boundaries, because spending $1BN to test a hypothesis with a potential return of $100K doesn't make sense.

Asking as hypotheses means framing every initiative as a falsifiable test with a clear decision rule. Avi shared a hypothesis card template that captures this thinking: "We believe that [change] will move [metric] by [MDE percentage] for [segment] within [time frame]."

The decision rules are straightforward: ship it, kill it, or iterate on it.

Cadence and capacity address the practical reality that teams need dedicated time and budget for experimentation. Avi was emphatic on this point, arguing that leaders need to set aside explicit experimentation budgets at the start of the year and protect them from being consumed by bug fixes and business-as-usual work.

Evidence systems round out the framework, covering how teams measure results and review outcomes. The goal is to move from emotionally based decisions to evidence-based decisions, and to make past experiment outcomes searchable so future teams can learn from them.

Learning from Booking.com and Netflix

Avi pointed to Booking.com and Netflix as organizations that have operationalized experimentation at a remarkable scale.

Booking.com runs over a thousand concurrent tests across products and segments, enabled by a thin central platform that prioritizes "enablement over gatekeeping." Their governance team focuses on coaching and tooling rather than approval, and they maintain searchable decision logs so that experiment rationale and outcomes are available to everyone.

Netflix took a similar approach in 2016, building an internal platform that removed the heavy lifting from A/B testing and made experimentation a core data science focus. The lesson Avi drew from both companies is consistent: if you lower the cost to try things, you'll increase experiment velocity.

He also shared a cautionary tale from his work with Anheuser-Busch, where Bud Light Lemon Tea was tested with a focus group composed entirely of Budweiser employees, who naturally praised it. The product cost $20 million before it was killed.

The fix, Avi suggested, would have been far cheaper: "You make a barrel of it, and you go to a bar, and you give out the first round for free, and you see if anyone buys a second one."

The discovery gap that agile hasn't closed

While Avi focuses on the velocity of experimentation, a presentation at CPO London by Lee Duddell, ex-VP Solutions at UserTesting, zeroed in on a problem that persists even in organizations that consider themselves agile: teams are still guessing far too often about what customers actually need.

The speaker presented research showing that 55% of product managers frequently guess when making decisions about their products, and he suspects the real number is higher. Forty-one percent of teams skip problem discovery entirely, and 62% never test high-fidelity prototypes with customers before building.

The consequences are significant. Up to 50% of the work that designers and developers do ends up being rework, with global rework costs estimated at $2.4 trillion.

Lee framed this as the central challenge for product leaders: teams are building experiences faster than ever, with more data than ever, but they're missing one critical perspective, which is genuine human insight gathered during design and development rather than after launch.

Five anti-patterns that undermine agile teams

The speaker identified five specific behaviors that contribute to this problem, and they're worth examining because they're so common.

The first is comfort with guessing, rationalized by the availability of A/B testing tools. Teams tell themselves it's fine to guess because they'll just test in production. But 70% of A/B tests are inconclusive, often because the team is testing the wrong thing or the wrong treatment. A large UK retailer halved their inconclusive test rate almost overnight once they started basing their A/B tests on real customer struggles rather than internal assumptions.

The second is rushing to solutions. The speaker shared an example of a B2B SaaS company that received many customer requests for a data export feature. They dutifully asked customers what format they wanted, built the feature, and launched it.

What they never explored was why customers wanted to export data in the first place. It turned out users were spending a day and a half each month building dashboards for senior leadership, a problem that, if solved properly, would have driven significantly more product adoption and willingness to pay.

The third anti-pattern is the HIPPO, or the “highest paid person's opinion.” Despite the promise of agile teams operating with greater independence, the speaker noted that many product teams still prioritize solving a key manager's issues over testing ideas with actual users. Evidence, he argued, remains the best counter to this dynamic.

The fourth is the perception that gathering customer insight during design and development is too slow to fit within a sprint. Lee acknowledged this was true when user research meant inviting people into a physical lab, but modern tools have made it possible to gather meaningful feedback quickly. And if you meet the customer's need the first time, the overall process is actually faster because you avoid the rework cycle.

The fifth is the misuse of A/B testing as a substitute for understanding. A/B tests can tell you which variant performs better, but they generally don't tell you much about your customers or why they behave the way they do.

Lee recommended that teams start by quantifying the rework problem in their organization, estimating how many sprints are spent redoing work that's already been deployed, and using that number to build a case for embedding human insight earlier in the development process.

Trivago was cited as an example of a company that adopted Teresa Torres' continuous product discovery approach, enabling product teams to run weekly discovery sessions with customers, which dramatically improved the impact of their A/B tests and reduced the tendency to rush toward solutions.

Prioritization as a customer-centered discipline

Filippos Koulyras, VP of Product at AIHR, brought a third perspective to the agile product mindset in his keynote at the Product-Led Summit in Amsterdam. His focus was on prioritization, and specifically on the gap between how product teams should prioritize and how they actually do it in practice.

Filippos opened with a scenario that drew knowing nods from the audience: you're in a roadmap discussion, the sales director wants one thing, marketing wants another, your manager just read about a cool feature, and everyone is pushing their priorities onto your roadmap.

Nobody is asking what the customer wants.

He called this the prioritization trap, where teams prioritize based on internal influence rather than customer input. At FedEx, where Filippos previously served as Head of Product, he observed a culture that celebrated shipping velocity above all else. Teams were rewarded for releasing features regardless of whether those features moved any meaningful metric.

Nobody was asking why we went live with this feature, he recalled. The incentive structure had decoupled speed from value.

Bringing customers into the room

Filippos' first principle is to use the customer journey as a prioritization map. Rather than looking at features or domains in isolation, product teams should examine the end-to-end journey from acquisition to customer support and use data to identify where they're losing value and where they're adding it.

This approach helps teams prioritize across features and across teams, allocating resources to the areas of greatest impact.

His second principle is co-creation with customers, and he was careful to argue that this applies in both B2C and B2B contexts.

At Adidas, where Filippos worked as a PM, the team had access to massive quantitative data from 20 million monthly visitors and ran extensive A/B testing. But they also visited customers in their homes, mapping their experience with the product in their everyday lives and traveling across Europe to gather insights from different personas and subcultures.

The combination of big data and direct customer engagement, he said, produced results that neither approach could achieve alone.

At AIHR, the team uses Gong to capture and AI-summarize customer calls, and every engineering team is required to shadow at least one customer call per month. The goal is to build organizational muscle around customer insight so that it becomes part of the company's DNA rather than something only the product team thinks about.

Frameworks as tools, not silver bullets

Filippos walked through several prioritization frameworks, including RICE, ICE, Kano, and MoSCoW, but he was careful to position them as tools rather than solutions. His practical advice was to operationalize one to three frameworks that serve different purposes, align definitions across the organization so that terms like "impact" and "effort" mean the same thing to everyone, and treat prioritization as an iterative, ongoing discipline rather than a one-time exercise.

The most striking part of his talk, though, was his discussion of culture. Quoting Peter Drucker's observation that culture eats strategy for breakfast, Filippos argued that prioritization frameworks will fail without cultural alignment.

He identified several failure modes: intuition overriding data, incentives that reward time-to-market over time-to-value, and what he called "performative prioritization," where teams go through the motions of structured prioritization while everyone already knows the outcome because leadership has predetermined it.

His closing advice was direct: If you're in an organization where intentions to change exist, but legacy mindsets are holding things back, the work of bringing data, involving stakeholders, and showcasing results is difficult but worthwhile. If the intentions genuinely aren't there, he said, "I personally believe it's time to start looking for a new job."

The common thread

These three speakers come at the agile product mindset from different directions, but they converge on a shared set of convictions. Speed matters, but only when it's the speed of learning rather than the speed of shipping.

Customer insight has to be woven into the development process continuously, not gathered after the fact. And the organizational culture, the incentives, the safety to fail, the willingness to let evidence override opinion, determines whether any framework or process will actually produce results.

The agile product mindset, as these practitioners describe it, is fundamentally about creating the conditions for teams to learn faster than their competitors. The ceremonies and frameworks are just scaffolding. The real work is building an environment where experimentation is cheap, discovery is continuous, and prioritization is honest.

Keen to discuss the agile product mindset with leading product experts?

Join us at Product-Led Summit Seattle on June 17–18, as Neal Rowland, Global Product Manager and Agile Coach at Stellantis, leads an interactive workshop exploring the agile mindset.

You’ll play an exciting, interactive board game that teaches incredible lessons of success and failure in managing products. Be prepared to laugh, challenge yourself, and have profound conversations with your peers and facilitator. The game is only the start.

This workshop is for all participants at all levels and all experiences. This will be the most fun you will have learning at a conference!

Will we see you there?

Secure your ticket

The agile product mindset: 3 practitioners' insights