During our Chief Product Officer Summit, Christine Tran, Senior Director of Product Marketing at Quantum Metric and Melissa Perri, Founder & CEO of Produx Labs, discussed how developing an Agile model can help build more customer-focused products faster.
Check out the discussion’s highlights below. If you wanna catch the whole thing, become a PLA member and get access to so much more. 👇
Christine Tran
We're speaking today about the need for speed, ways to improve speed of market and customer focus.
Melissa, in your book, you talk about the early lessons you have had as a product manager, which are still relevant to product managers starting out today. Can you talk about your journey?
Melissa Perri
When I started up as a Product Manager, I joined a place called Capital IQ. We operated in a very typical product management style for the time, which was waterfall. My job was to go to all the salespeople and all the people who talk to the customers, write down long lists of requirements, bring it back to the engineers and write out spec documents, ship them off to the engineers and pray that it will come out one day.
When I ended up going to a start up a couple years later, I had a very rude awakening, where our engineers did not want to read my 30 page spec documents. They just built what they wanted to build and we were struggling to figure out how to work together. That was the first time I was introduced to Agile. We started to work together, learning Scrum, trying to do some basic Agile principles of just collaborating. We didn't follow a formal process and we didn't go to any training but we sat down together and figured out how to build better products.
However, I experienced what I call the “build trap”, where I realized it didn't matter if we were Agile and using all these processes or working together but it mattered if we were building things people used and we weren't. I got on this journey to figure out how we can make sure we're building what the customers want. I started to incorporate lean startup principles into my techniques and started interviewing customers, running experiments, setting success metrics, and then iterating until we found something that hit our goals.
When I left that startup I found out that a lot of places didn't do that, so that's how I got into teaching and helping others understand this form of product management. I went on to show them that you had to make sure you were building things people actually wanted, it wasn't just about following Agile to a tee, going to a training and getting a certification in Agile and being done with it.
Christine Tran
When I first heard about the Build tab, I thought it was such a powerful metaphor that resonated with product managers immediately. It's something that a lot of people are feeling the pain of.
One thing I've heard people say is that to be a great product manager, you have to make better bets. You're always doing this because you're always iterating your products, releasing new features and each time you have to get better at what you're delivering to customers.
How do successful product teams make better bets? What are some actionable insights you can give to the product leaders?
Melissa Perri
The first thing is understanding what your goal is and what you’re trying to achieve, because you can't place a bet against something, if you don't know what the outcome should be or what you're trying to achieve. That's where a lot of companies get stuck because they don't talk about what's the end result and what's the success metric.
Sit down with your stakeholders and your team and have a conversation. You don’t have to redo the entire organization, change the culture and make everybody focused on outcomes, but one small thing you can do to get started that you have full control over and that also helps you placing better bets, is getting to the outcomes of what's going to solve the problem for the customer, but also what's going to solve a problem for the business, write those down, and then build the thing. When you ship it, check it. That's how you know you have real success metrics.
The teams who place good bets, they're always oriented on what the outcomes are going to be. It shouldn't be a million bet,you want a couple finely tuned to the problem you're solving. Once you have the success, you have to understand the problem. Good product managers will research, talk to the customers, watch them, and they don't just listen to the customer and make the customer solve their own problems, they utilize technology in ways the customer never even thought of to solve the problem.
What's going to help you set good bets is a combination of understanding where you want to go and what those goals are and deeply understanding what the problem you're trying to solve is. You want to make sure there's some guardrails around those bets too because if it's something you can't possibly build with your company, that's not a good bet. It's not about quantity, it's about the quality of bets.
Christine Tran
You just had so much that I'm going to have to unpack. You were touching a little bit on Product Kata, which is a neat application of an existing metaphor to help teams iterate in smaller batches, learn and optimize for its outcomes.
What if the outcomes aren't neatly framed like that? For example, there’s two personas you're working towards and you're trying to evaluate that they're both underserved. How do you figure out which one to pursue next when you're given the goal of just growing and scaling quickly? Is there any lightweight way to figure this out without dedicating a lot of engineering time?
Melissa Perri
Product Kata is a way to systematically walk through a series of experimentation to hit a goal. It's all about learning, and it comes from Toyota Kata. What Toyota would do in their lean manufacturing systems is always have a continuous improvement mindset. So Product Kata really looks at continuous improvement for Product Management.
The basic steps of Product Kata are:
- What's my goal?
- Where am I now, in relation to that goal?
- What's the next thing I need to learn or do or experiment around to get closer to my goal?
- Run an experiment or design experiment to do that.
- Reflect on it and see if I got any closer to it.
The first step in Product Kata is you have to set your goal. Before you get into it, this is where we start to deploy and talk about great product strategy and coming up with the product strategies, what those outcomes are and what those goals should be. The goals for the C-suite are going to look very different to the goals for a product team, but they should all ladder up to each other.
If you’re getting into Product Kata and you don't know what goal you're orienting towards, my first advice is look at what's above (e.g. what's the goals for the leadership above you or for the company), then think about what can you do to help hit those goals (e.g. what's the problems I need to solve?, what do I think are going to be the success metrics to show me I've solved them), and then that should be the goal you're going to use for the Product Kata. So you're going to have to do some work to first clarify what those goals are, and what's the direction you need to be experimenting towards.
I've seen a bunch of teams pick up Product Kata without clarifying the goal and what happens is they run a bunch of experiments, but they don't know where they're going. This is all about placing good bets as well. You have to clarify what the goal is first, that's half of the battle, then you can jump into experimentation.
If you’re trying to choose which of two personas you should pursue next, you should do the research on it. If I was looking at this from a product strategy perspective, I'd think about how do these personas use my product differently, which one might be a buyer or a user, how do they impact the economics of our business, which one has a stronger problem, how will they impact our business, what are the goals of our business, etc. Which one's going to further us closer to our goal, and that doesn't mean you don't solve problems for both of them, but you're probably going to prioritize one over another.
Break it down and understand deeply, how did these personas use my product, why are they important and then how can they help me achieve the success metrics I need to hit with my product.
Christine Tran
Let's pivot a little bit. You've got your strategy defined, you're experimenting, you're iterating.
One of the things I've heard recently is that hypotheses are great, but testing your hypothesis is even better. Do you agree with that? How do you find the balance between those two things?
Melissa Perri
Hypotheses are meant to be tested. Starting with the hypothesis is important because it communicates to the company and the teams that we have to pull some data experiment to try to figure out what's going on. That's half of the battle.
Then, the other half of the battle is designing a good experiment to learn quickly, if it's in the right direction or not. Experiments rarely ever come out incredibly black or white. After an experiment it’s never 100% definitive that if we launch this feature, we're going to make $200 million in three years, for example. However, what the experiment should do is make you feel more certain that you're going to hit the $200 million. Maybe we're starting at a 10% certainty, we run some experiments, get really good customer feedback and now we're at 60%.
Product management is about thriving in ambiguity, our job is to basically make decisions with imperfect data, but the more we can get confident about the data, the more data we do have, the closer we can get to feeling more certain about the right way to go. Running experiments, gathering the data, looking at it, crunching it and figuring out where to go makes us more likely to succeed rather than the opposite.
Christine Tran
We talk a lot about econometric, it's really about learning faster and what helps you with speed and market.
No matter the size of your organization, the bigger you get, the more silos you have. People are coming to meetings with different sets of metrics they're looking at.
How do the product leaders help get everybody aligned on the same page? At the end of the day, that's what slows down teams and you're not able to bring the market faster when there's no fast decision and faster learning.
Melissa Perri
As you scale a company, your products get more complex, your strategy gets more complex, your organization also gets wildly complex, you need different roles, you have different teams, product lines, features going out, and everybody's working on different things at the same time, so it becomes more important than ever to bring people together to look at a system of record of metrics. This is where the outcomes become particularly important. All the different functions will have their own set of metrics, but you all have to be aligned around what is the outcome we're trying to achieve from all of these metrics.
Product has to look at outcome focused things, but then they also have to tie in all these engineering metrics, ops metrics to make sure the health of the product or the health of the team is able to fulfill that. For instance, a good metric from technology teams is bugs (e.g. how many bugs do we have currently outstanding on a product). That's not going to tell you if you are successful, but it will tell you why people might not be happy with your product.
Product needs to make sense of all these metrics, put them in the right places and use them at the right time. Data can tell us a lot, but it won't tell us a whole story, so when we do see a metric, we have to ask ourselves ‘why’ and dig deeper.
Christine Tran
Do you feel like there are metrics people over rely on and are overrated vs. metrics people under rely on or underrate?
Melissa Perri
I think NPS is a metric everybody relies on too heavily. I also don't like the way it's worded “would you recommend this to a friend?”. This is bad wording because what if my friends don't do product management, I'm not going to recommend it to them. The way you phrase those types of questions is incredibly important. If you phrase it the right way, you can get great data.
NPS and click throughs are what we call ‘vanity metrics’, which are metrics that are not showing you if something was successful or not, not showing you an outcome or something you can compare month to month and have a decision made from it. Those metrics make us feel good, because they’re always increasing, but they're not giving us any action.
The mark of a good metric is one that allows us to take action and make a decision, and if we don't have metrics that allow us to do that, we should be using them to help us inform our health which will ultimately lead to an outcome.
Christine Tran
What is the one piece of actionable insight you could share with product leaders?
Melissa Perri
Find out what those outcomes are. A lot of people get really ambitious and think they’re gonna change everything, and they run into a wall. Especially if you're part of a larger organization, I promise you that you're not gonna be able to change the entire culture of a large organization, by yourself in a day.
However, what you can do is make a very meaningful impact and you can start to be the person other people look towards to model good behavior, if you start small and you start to get buy-in for it.
Look at what you're building now, and ask yourself “do we have success metrics?” and if you don't, start a conversation with the executives, stakeholders and your team and ask them “what do we think is going to happen when we ship this?”, “is that success?”, or “what would you like to happen?”.
Those conversations start a whole conversation about what we are doing:
- Are we measuring success?
- Do we need to measure success better?
- Do we have the right tools to do this?
- Are we building the right thing?
This is a conversation that will grow and help get you the change you want to see in these types of organizations where you're defining success and working towards outcomes and building things people actually want. It's a simple thing to do but I've seen it change companies and change the way people talk about their products.
Christine Tran
What's one thing you know, now that you wished you knew 10 years ago?
Melissa Perri
One thing I know now I wished you knew 10 years ago is that people are everything when it comes to organizations and businesses. When we go to school, you must learn how to do math and science, when you go to college, it's all about the hard skills.
I never gave a lot of thought to the soft skills of influence, negotiation and trying to win people over. I didn't learn that until I got into my career and started hitting walls there. Then, I got curious and decided to get better at it. How do I build people up right with me and take them on the journey with me?
That's made me way more successful than I ever could have been without it. I wish somebody would have stressed that at the beginning. I actually don't think they're soft skills, I believe they’re core skills.
Christine Tran
How do you cultivate that skill?
Melissa Perri
I treated it just like an experiment, the same way I do with product management. I was in organization consulting, trying to get executives to move in different ways so I'd try different tactics. I started to try to find out what was working and what wasn't, how do I change my approach, my tone, and then also listening to others, getting to know them more, understanding their goals, just the way you would want to do for a product.
If you apply that to people as well, it opens up this whole world of humans you may not have been privy to before. Everybody's got their own goals, their own agenda, their own things they want to achieve, and if you know and can help them achieve those things, then you're gonna get your agenda pushed a lot further.
Wanna learn even more insights and invaluable knowledge from product leaders?
Join us for a comprehensive 6-week program, packed with all the knowledge and insight you need to shape your career and supercharge your leadership skills in the PLG space.👇
