[On-demand] Product Management Webinar: AI and Productivity

[00:00:00] Janna Bastow: Hi everybody, and welcome. Come on in. Come on. We’re just kicking off here. We’re just getting settled in, so why not jump in, find your way to the chat, and say hello. And we’ll be kicking off with myself and David soon

Everybody welcome. Come on in. Come on in. I can see that there’s people coming in now. Jump in, say hello, let us know where you’re calling from. Let us know where you’re coming in from. Oh, I can see Jonathan coming in from Austin, Texas. We’ve got Beryl from South Africa. Oh, hi, Nils. Friendly face. Good to see you here.

Thanks for joining. I’m sure there’s lots of other people here just finding their way to the chat. Also while you’re at it, open up the Q&A, and if you’ve got any burning questions already, you can put them in there. Or, you know, as the conversation progresses, drop your your questions in the Q&A.

You should be able to see all the questions as well as up vote on them. Get your questions in early, and then we’ll be able to see which [00:01:00] are the key ones that people really wanna tackle. We’ll try to tackle all questions today, but as you know, these things get really busy as everyone starts chatting.

We’ll figure it out as we go. And I can see people coming in from all over the place now. A couple of people from Atlanta. I used to live there. Somebody from Toronto. I grew up near there. Somebody from Salt Lake City, Afghanistan, Boston Birmingham. A real mix of folks here.

This is great. We’ve got Matthew Erickson, a real human here, reporting in. I wonder if there’s any AI people listening in just yet, or are we not at that point yet? David and I were talking about this before we started, was the fact that, you know, it’s great to be able to have real-life conversations.

We’re seeing so much out there that’s conversations that could have happened, but they’re made up by AI, right? And it’s all just this content that’s being created. Whereas these are real conversations where we get real insights and I think it’s really key to have this to keep talking about this sort of informa- this sort of thing.

Nils just said, “I’m AI augmented,” “but still a real human.” Yeah, I think we can all say that we’re probably AI augmented to some degree, right? We’ve been using AI-type [00:02:00] tools for years, and it’s obviously just accelerated. So today’s conversation is gonna be about what can we make with this, right?

You know, AI’s making us faster. We’re doing lots of stuff, but is it actually helping us reach our goals? Is it making us better at what we do? And I think it’s really important to keep asking these questions. I think everyone’s We’ve got enough people have settled in, which is great. Thank you, everybody who found the the chat.

Feel free to make use of the chat today. Talk to your fellow product people. Connect with your fellow humans. Drop in your LinkedIn if you wanna connect with fellow humans as well. If you’ve got any questions, drop them into the Q&A section. I will be trying to keep an eye on the webinar chat, but it goes pretty quickly.

Drop into the question section. That way we definitely see them. And up vote other people’s questions, that way we know what people really wanna chat about. In the meantime, I think we’re just about ready to kick off officially

All right. So welcome everybody. Welcome. Come on in and we are ready to kick off today’s webinar, which is all about how AI made you faster, but did it make you [00:03:00] better? We’re here with David Pereira, who’s gonna be chatting to us about this. These webinars that we run are a series of expert firesides that we run.

It’s usually a combination of either firesides, like today’s gonna be, or presentations from experts. But it’s always with a focus on the expertise the experience, the insights that these product p- people, these product experts bring to the table. You’re always gonna have a chance to ask questions.

These things are always recorded, so you can see a history of all the ones we’ve done in the past. And for today, it is gonna be recorded and shared with you, so you’re able to share it out with your colleagues and with your peers. As I said, drop your questions into the Q&A, drop your chat into the chat and you know, we look forward to to having this conversation with you. Before we jump into the meat of things I just wanna say really quickly what we’re about here at ProdPad.

I see some familiar faces here, so some of you have been longtime supporters, so thank you so much for that. Some of you might be new to ProdPad. ProdPad is a tool that myself and my co-founder built when we were both product managers ourselves. You might know me and my co-founder [00:04:00] Simon from the Mind the Product series of events and community.

Basically ProdPad grew up in and around that. We were surrounded by all this product expertise and put that to use to build a tool that actually helps product people run their product org the way that, you know, we should be doing. This is my cat Orion. He loves to Zoom. And so we built ProdPad because it helped us get a sense of control and organization.

You know, when you’ve got all these different ideas and experiments flying around, you’re trying to connect it back to your your overall goals and make sure that everyone’s communicated to well. But also provides you the single source of truth for all your product decisions. It’s the type of tool that gains value over time.

You can look back on it and say, “Actually, you know, have we tried this before?” Or, “What customers have said things like this?” Or, “How does this connect with this goal that we’ve been trying to reach for a couple years?” It’s actually a tool that you can try for free. You can see how it works with example data in our sandbox.

You know, you can see how lean roadmaps, Now-Next-Later roadmaps and OKRs and experiments, they all play together in one [00:05:00] space. And our team is made up of product people. You know, it was founded by a couple product people, myself and Simon. We’ve got lots of product-minded folks who’ve helped build it over the years.

So give it a try, and let us know what you think. We love to hear from our fellow product people. In the meantime, let’s jump over and introduce you to David. So my guest today is David Pereira. David’s been in and around product for 17 years across e-commerce, automotive, public services, and he’s done everything from engineering through to running companies as a CEO.

He wrote this book called Untrapping Product Teams, which is a bestseller in over 70 countries. And these days he coaches product teams and leaders and has some very sharp opinions about what AI is doing to the product practice right now, which is exactly why I wanted to have this conversation. So David, welcome.

Everybody say a big hello to David. 

[00:05:50] David Pereira: Hello. Hello. Thanks for having me here. I’m really looking forward to this non-AI chat. 

[00:05:59] Janna Bastow: Yeah, [00:06:00] exactly, right? If we had this conversation three months ago or three months from now, it would be a different conversation, right? Yeah. This is such a fast-evolving space.

We’re just grappling with this new, I don’t even wanna call it tooling, these capabilities that have been opened up to us as product people. You know, not just in terms of how it enables us to do our jobs, but how we’re applying it to our products to enable other people do whatever jobs we’re helping them to do.

Now, you mean you wrote this book the Untrapping Product Teams. You know, what’s… Just a little quick update on this. I mean, what’s trapping product teams today versus when you wrote the book? 

[00:06:38] David Pereira: This is interesting. Like, when I wrote the book, what was trapping teams was mainly the systems they were operating.

Pleasing stakeholders, running extensive roadmaps, extensive backlogs, and focusing on output, and all of the things we know. And then decision-making very hard, like lack of [00:07:00] strategy, which led to meeting marathons because you had to discuss everything so you can agree on something. But today, when I look, these things are still real.

They’re still present in the world, and what is the trap right now is believing that using AI makes us better immediately. 

So this is something I am seeing teams like say, “I am AI empowered, and I’m delivering more.” But then I always say “Okay, but what for? And how did it improve?” ‘Cause the biggest trap I see today is a lack of understanding of our foundations and how that helps us do what we’re supposed to do, create value for customers and business.

[00:07:48] Janna Bastow: Yeah, absolutely. I mean, all that stuff that you outlined that was trapping product teams, you know, teams spending too much time creating big roadmap documents and backlogs and struggling to communicate to stakeholders. All of this stuff is actually exasperated [00:08:00] by having AI, right? It’s way too easy to create, you know, a massive spec saying what it is that you’re going to do or you know, to miss the mark in communicating because you’re not actually even paying attention to the nuances of what you’re writing and how you’re communicating.

So you know, you could say that all of this is actually made worse by AI even though there are some things that have been obviously made better, right? We’re able to- … create as much as we want. We don’t even have to write the spec anymore. We can just, type out something really simple or just voice chat and it creates something for us.

But doesn’t mean that it solved any of those problems that were trapping product teams. 

[00:08:36] David Pereira: Exactly. Yesterday I gave a lesson at one school and it was about backlog management, and we touched upon exactly this topic of specification. I remember when I started my first assignment as a product manager, it was 2012, and then someone told me, “Your result is as good as the product requirement you can write.”[00:09:00] 

So did I. I tried to write requirements as precise as possible. I wrote things of 30-plus pages and so on, so the engineers could implement without talking to me. Of course, they would not understand the problem they were solving, but they could code. And then within time I was reflecting on the journey.

Within time we started working on smaller chunks, like creating user stories and so on, and iterating, focusing on learning faster and adapting on the way. And after we started running Discovery and we said, “Okay, let’s embrace the unknown. Let’s figure out what works,” and so on. And all of this requirement extensive were left behind.

And today I stumble upon something called specs-driven development where- … you use AI, you give a little bit of context, and then AI will create a specification for you to use another AI tool to code for you. This… I don’t feel that this is evolution. I feel like I’m traveling back in time just [00:10:00] doing the same thing faster.

[00:10:03] Janna Bastow: Yeah, absolutely. I mean, in all of that, where is the customer? It’s, it’s- Yeah … easier than ever to write a 30-page, 80-page spec. And yet if it doesn’t actually see the light of day, doesn’t get any of that feedback, that face time with customers, then what you might be creating, even though you can do it faster, doesn’t mean it’s any better.

And so I think this is one of the things that’s happening is we’re so enabled to create right now that we are not taking that step back. And it actually reminds me a little bit of, when mobile came out in 2007 right? We had the ability to create apps, and people would just create apps regardless of

whether they actually solved problems. It was just a bunch of stuff created. And some of them pulled ahead, pure luck. Some of them, obviously most of them, died a death, right? We started realizing that not everything needs to be a mobile app. Not everything. You know, a lot of these things were just features rather than standalone marketable apps that could actually survive.

Same thing [00:11:00] happened with with web apps as well, right? I mean, we created websites for everything. We’ve done this multiple times, right? Multiple different cycles. And we realized that actually, you know what? Just because we’re able to get something out doesn’t mean that it’s actually the best thing to do.

And I feel like we almost got there as a product world as the craft. There was this move away from just creating things for the sake of it, and actually, we were talking about discovery. We were talking about, you know, making sure the customer’s at the center of the conversation.

And then we got AI, and we forgot all the lessons, right? We just went, “Ah, screw the customer. We don’t need to talk to them. Let’s just build as many things as possible.” And this is just like when we created all those little apps and everything like that. Some of the things we’re creating, just by chance of, having the opportunity to create so many things, some are gonna win.

But the chances that your thing is going to win is extremely small unless you’ve actually tied it back to a need a market requirement to something that’s actually gonna support this app going [00:12:00] forwards. 

[00:12:02] David Pereira: Yes, and this connects to something I have been hearing from many C-level. Yeah what I heard recently, discovery is our main problem.

Discovery makes us slower than we should be. I do know that- it’s, how teams run discovery is very different, but the main objective is to drop bad ideas fast enough, so you understand what’s not gonna fly, so you don’t invest your time there. And then you uncover something very important, what we don’t know.

Because the reality is like this, we don’t know what we don’t know, so we better start stepping to the unknown so we, we uncover that. In a recent talk with a C-level, what happened was like, we want our team to stop doing discovery, and what we want is to use AI. And for example, we use AI to interview product managers to create these specs, and then we use AI to code.

I said you can do that. Of [00:13:00] course you can.” Technically it’s feasible. But the real question is, should you do that? And how will that create something that customers choose to use? 

[00:13:11] Janna Bastow: So interesting that you’re seeing C-level reacting to the fact that discovery, you know, isn’t, is wasting their time, right?

I mean, you know, you might be able to see from their point of view that it doesn’t look like productive use of time, right? It’s not the creation of something. They can’t sell discovery. Discovery is there to make sure that they don’t spend their time building the wrong thing. Yes … but those costs are quite distant from what they’re seeing, right?

And I feel like there was… We were on the path to get there, … we’d assorted getting C-levels on board with the fact that actually it’s not about how many lines of code we do, right? It’s about whether the problem is actually solved for the customer. Let’s measure based on the outcomes rather than the output.

And all of a sudden we’re swinging straight back into you know what? We can output more faster. We can get more lines of code. We don’t need to do discovery because we can just do this 20 [00:14:00] times. And I don’t think they’re really realizing that the value of discovery isn’t just helping you find out what that problem is to go solve.

It’s helping you prevent… It’s preventing you from creating a whole bunch of stuff and driving your company down a path that isn’t valuable for it, right? So let’s say you create 20 prototypes. Okay. So let’s say you get those 20 prototypes out in front of people. You know, by the time you’ve actually gotten to the point of determining which of these paths is the best one, you’ve wasted so much time creating a bunch of stuff that actually you could have discovered right up front, half of which, 90% of which weren’t actually the right way to go.

And so there is this value in discovery, which is not creating a bunch of stuff-

not actually solving the problems for the business. And I think happening where, you know, put it this way, we’ve got all these companies who are talking [00:15:00] about being, run by agents, these solopreneurs who are building up these companies ’cause they’re completely agentic run.

Cool. But are any of them actually successful? I mean, yes- … they built lots of code. Yes, they are doing it for cheaper than a team of 10 real humans. But, right now none of the apps on my phone, none of the products I use were built by agentic companies, and I don’t hear of anybody actually using these things.

I just hear people creating these things, which is a big difference. And actually there’s a couple of interesting comments in the chat. Matthew says “Optimizing for more stuff doesn’t mean you automatically create better stuff.” Yeah, exactly, right? And Melissa follows up with, “Speed is seductive and difficult to argue against before you’ve learnt the lessons of going the wrong direction” 

[00:15:50] David Pereira: Yes.

And connecting to, to this part of agentic and so on, recently I talked to founder from Stockholm, and the founder shared with me [00:16:00] the challenge they have right now. They are running a lot of things with agentic AI, and said, “I need some help to activate our new functionality because customers refuse to use.”

And you know, what you want is not always what you need. What he was asking for is, “How do we get customers to use what we’re building?” He said, “I have a feeling we’re building way faster than they can use it.” And then I said, “Let’s step back a little bit.” How did you choose to build that? And then he said, “I didn’t.

Our AI agents chose that.” And what is happening, like discovery had other advantage that is often ignored, is the knowledge you build within the process. 

[00:16:42] Janna Bastow: Yeah. 

[00:16:43] David Pereira: It’s very similar to strategy. Recently, I had worked with two product managers, same organization, and one took quite some time to craft a strategy, and then used AI to shape the visualization of that to make it more digestible.

But the thinking [00:17:00] was clear. And the other person crafted very fast and created a strategy, present it to everybody. Everyone was shocked. Oh, okay, it seems solid. But the thing that happened is that the first one managed to lead strategy across departments and could get people behind, ’cause the thinking was clear.

The other strategy became a document everybody ignored, ’cause no one knew how the thinking behind it happened, ’cause it didn’t happen. 

[00:17:31] Janna Bastow: Yeah. Absolutely, and this actually aligns really well with what good roadmapping is. There’s all these bun fights out there around what the best version of a roadmap is, what a roadmap should look like.

And, even as the person who made Now-Next-Later a thing I don’t care, right? It’s not about what is on your roadmap, it’s about how you got to that roadmap, right? So it’s not about the roadmap itself, it’s about the roadmapping process. It’s about the process of just getting out some basic assumptions and then checking those assumptions and improving on [00:18:00] them, right?

It’s getting people aligned around it so that when you show your roadmap to somebody, it’s not this big surprise, going, “I’ve never heard of this before,” right? And this is the danger with AI, is that we’re shortcutting that. It makes it super easy to, and for everybody in the team to contribute to this.

You know, one person to come up with a document that’s too long for a human to bother reading of their assumptions of what they think they should do. And other people in the team to feedback. They often summarize stuff, and they add their feedback or whatever, but they haven’t really internalized it.

They haven’t read it themselves. And so what you get is a bunch of missed assumptions, sometimes just slightly- … missed assumptions, sometimes massively missed assumptions. And AI’s really good at being confidently wrong about stuff. You know, sounding really good, but actually when you dive into it, you scratch under the surface, you’re like, “Okay, this looks like a really well-presented document.

I get where you’re coming from. And eight of the 10 bullet points here all make sense, but two of them, oh my God, it’s completely misunderstood our constraints and our market and our customers and everything else.” And [00:19:00] so oftentimes these things will be built on, and people won’t catch them because it’s not people checking it.

It’s their AI system summarizing it and feeding back on top of it. And here’s the thing, is like AI could actually be brilliant if everyone used it with a certain sense of discipline But what’s happening is it’s almost too seductive, right? It’s this shortcut. It’s too easy to not read the big, long document.

It’s too easy to take your thoughts and see that it turned it into this nice presentation and not really spend the time diving in and criticizing each individual line. And that’s the critical thinking that we need to pull from the people who are working on this stuff. And if critical thinking is just being scraped off here or there, the little shortcuts being taken, the ultimate plan that’s being created isn’t battle tested.

It’s not being sense checked by other people. It’s not turning into something that actually is going to likely solve problems. And people look at it going, “Ah, but this looks better than what the last guy did,” right? Because, the last guy didn’t turn [00:20:00] into a beautiful big, long presentation.

It looks interesting. It looks like it’s good, and therefore people assume it is good, and it’s not. It’s a facade that AI puts on it. 

[00:20:10] David Pereira: And how we can use that. That’s a thing. A lot of people now are talking about faster. Which technically it is possible, but is it the right place to look at?

I like thinking about better, like looking at the possibilities AI enables that were very hard in I would say very recent past, like a year or two years ago. I always look at how you mo- how fast can you move from idea to value? And the thing is, most people think it’s a linear process, which is not.

What is gonna happen there is a lot of fun. So you will come up with some assumptions, and then they turn out to be false, so you need to pivot, you need to do some things, and then you’ll come up with something that makes sense. But throughout this process, one thing you have to do is [00:21:00] to experiment with different solutions for the same problem.

[00:21:03] Janna Bastow: Yeah. 

[00:21:04] David Pereira: And some solutions are gonna work, some won’t. The ones that don’t work, you drop them. The ones that work, you start investing more. This part of exploring with multiple solutions, I struggled my whole life to get teams to decide to invest into that, because they always had more to do than capacity.

Yeah. So they would not do this. And today, we have this possibility. We can use AI to explore with multiple solutions and really compare. But teams are still not doing that. They are still choosing to build more solutions without this exploration part, because that is something that could help us be better.

Not necessarily faster, but better 

[00:21:45] Janna Bastow: Yeah, absolutely. And actually, I think there’s something really in there, right? So there’s this nuance between being able to you know, you can generate, you can create solutions and you can visualize them. That’s really powerful, right? If a picture is worth a thousand words, then a prototype’s worth a million, right?

And [00:22:00] we know this. This is why we’ve gotten good at prototyping. We’ve built really good prototyping tools and it’s still a time-consuming thing up until recently. Now you can create those prototypes. You can create as many of these things as you like. But I think the thing that we’re shortcutting is on both sides of that is one, it’s we’re not spending enough time in the problem space.

It’s too easy to generate a prototype and go,”Great, let’s iterate on this. This is 80% of the way there.” And actually, that prototype might have missed some fundamental problems, right? So you’ve jumped to a solution. You’ve jumped ahead. And I think there’s something in the prototyping process where you start discovering problems.

Because you’re building it up slowly, you’re checking it as you go. Whereas if you could build a whole prototype out, it’s almost as good as the live app could be, in theory. It looks like it’s a good app. And so we’re almost skipping the prototyping and going straight to the picking one of them as a solution.

Now, if we actually had the discipline to say, “Actually, let’s really think back to this, the problem space.” And then we can build a prototype, but we’re willing to throw this whole prototype out, right? We’re willing to train our AI [00:23:00] to actually ignore that entirely and then come back to this problem space and now let’s come up with something a different direction and come up with something a different direction.

But because it pulls us along to this solution and it’s so confident about creating something that looks good, we skip it and go, “Ah, close enough. We don’t have to go do the discovery thing. We’ve already got something that works.” And I think the other piece that’s getting missed is the actually checking that these prototypes work, right?

Now I am seeing people out there using synthetic users, right? By that I mean, I see your face. That’s the face I made too when I heard about it, and every time I see it happening. Because they are building something that AI says is gonna solve their problem, and then instead of taking it to humans, they’re taking it to AI to pretend it’s a human to tell them what they think about it.

And I mean, some people immediately see the problem with this. Other people go, “Oh you know, it knows a lot about a lot of things, and therefore it can provide these insights,” and I’m sure there probably are some interesting things that can [00:24:00] come out of it. But, what is the point of us saying our entire lives as product managers, we need to get out of the building, we need to go talk to people, but we’re too busy to do that because we’re busy writing communication docs and reordering backlogs and making roadmaps and stuff like that.

We’re too busy fine-tuning a prototype to get out there and talk to the customers. Now, we’re not too busy doing all that. All of that is just done like that. So we should have all the time in the world to get out there and talk to the customers. I swear to God, product people often just don’t like people because now they’re finding any excuse not to talk to people, even when they have the time.

If there’s ever a time to get out of the building, it’s right now, because you have time. Something else is doing the rest of your job. The hard part is connecting to people and asking the right questions and dissecting those answers 

[00:24:47] David Pereira: And that’s totally true. And about the problem space, what I keep saying ’cause now it’s prompt away from idea to a working prototype.

It’s just a prompt. That’s what is happening. It is seductive. [00:25:00] But what I say is, all right, so who is the audience and who is not the audience? Let’s start at least with this. Who is the audience- who is not, and what are we doing for them? And then we need to ask the questions, what are we assuming to happen here?

We are assuming they have a few problems. We are assuming these kind of things and so on. Just recently I had dinner with a product lead in Germany, and he was telling me about confidence, and he took the confidence meter from Itamar and he adapted to his scenario. And I see very much necessary that today.

And he said, “From one to five in a confidence meter, it is what we believe. We beli- we only believe. And then from six to 10 is what we start knowing.” For example, if your idea, you are the only person who has ever heard about it, then confidence cannot be anything more than one. If you talk to someone and the person start [00:26:00] nodding, it’s a two.

If you interviewed customers and you identified a pattern, then it’s a three. And then you start going from there. And when you start thinking about this, if he we… If we would start naming what we assume to happen, and then identify which assumptions we do have evidence and the ones we don’t, and then one, which are the ones really important, if they are proven false, our idea is worthless, we could select these assumptions and say, “Let’s build a prototype to test these assumptions.”

Because today we try building a prototype to put in the market, which is a very weird idea because it is just too far. I have been telling people there’s a term MVP I don’t like using because it became everything, first version and so on. So I prefer using minimal testable product and say, “Okay, let’s look at the assumptions.

We lack evidence. Let’s build something to test these assumptions.” And then once we learn it works or doesn’t work, we decide what to do. Then after that, we need to have something that customers choose to use frequently, [00:27:00] but is small. So this learning we will build throughout the way is what will enable us to create a product that users choose to use.

That was a conversation I had with the founder in Sweden. I said, “It is not about shipping faster.” If you overlo- overwhelm people with functionality, they don’t know what it is a need for them, they will not choose to use it, and they will choose to use it once you understand them before, and you do something that is natural, not rational.

[00:27:29] Janna Bastow: Yeah, absolutely. And I think there’s this whole assumption that people still lean on, which is that if you build something, customers will come. And I think we have to recognize that- We’re now at the point where we can build, anyone can build anything, right? And if you’re able to build something in a weekend if you’re able to come up with an idea and build on it there’s nothing setting other people apart from building the same thing themselves the next weekend, right?

And we actually have a real opportunity with AI, which is to up [00:28:00] our expectations of quality, up our expectations of what we are delivering in terms of quality, right? AI seems to do a really good job of creating apps that are, like, 80% of the way there but have a bunch of funky flows, right? Everything is just not quite solving things the way that you would if you really thought through that process.

But instead, we’re saying, “Actually, 80% is good enough. Ship it. I’ve created this new app. Get it out there,” right? Build it, and they will come. And actually, I think customers will start recognizing that there’s way too many apps out there. There already were too many apps out there that were kind of junk, and now there’s way too many apps out there, and they…

the problem is that they all look the same. But no one’s really paying attention to the flows from identifying that problem, making sure the person understands that this is what this thing’s gonna do for them, and then really thinking through that customer journey all the way through to the end.

And that’s only really solved when you really watch people do stuff over and over again, when you really truly understand what their problem is and how you solve that particularly unique problem that they’re trying to solve. And that’s the [00:29:00] discovery process. That’s the iteration process, and we have the opportunity right now as product people to up our expectations and go “Yeah, you know what?

We made 100 prototypes, and we threw out all but one and iterated on that, and here’s why. Here’s the discovery that we did alongside this prototype to get there.” And we could be ending up with products that are 100 times better or 10X better as opposed to 10X the number of products. Instead of just creating 10X the number of things but not actually curating to pick out and create the best of it using these tools that speed us up.

[00:29:33] David Pereira: What I see is like AI is the amplifier. So if you have good things going on, it can make it better. But if you have foundations that are getting in the way, so bad things become worse, just faster. And I keep thinking … Yeah, so that’s what I’m seeing. And I keep thinking about an experience I had as a product manager.

It was long ago, but we were selling wine in Brazil, and [00:30:00] Brazilians overall don’t drink wine. Brazilians think wine is too expensive and so 

on. 

And the idea of this company was to make wine simple to drink and fun. So it was cool. And I was part of this and so on, and it was very interesting because we were doing the design classic inside out.

So we had some assumptions why people are not buying, so we were trying to make something cool. For example, you say the meal you’re gonna have and then we recommend the right wine for you, and nothing would happen. We would change the product detail page, nothing would happen. The product cards- nothing happened. People kept not buying. Then one day I went to the supermarket, and then I saw the, those amount of wines there. And I said, “I know why nothing is happening, because we are not understanding how people choose to buy wine.” And then I talked to one researcher. I said, “Hey, what if we start going to the supermarkets?

We stand there, and then we see how people choose wine, and then we [00:31:00] talk to them.” And designer said, “This is actually not a bad idea.” Said, “Let’s just do it.” And said, “Let’s see if we can get approval.” I said who cares about approval? The supermarket’s just to the other side of the street. Let’s go there.”

[00:31:13] Janna Bastow: Love it. 

[00:31:14] David Pereira: And then we went there, and funny things are happening. There’s a guy selecting some wines there. He was wearing a suit, and then he picked the wine and so on. Then we approached him, and then I asked “Could you tell me what made you choose this wine?” He look and said, “You know, I have a date tonight, and I want to impress a girl.

And I learned that this grape is nice. I like the label. It looks beautiful, and it’s the only grape I ever know. And the price seems decent, so I chose this.” There was nothing to do with all the things we considered. Then we got to another person and so on. It was very random things, and we were not designing for any of them.

So for sure- … [00:32:00] we would never increase what we were trying to do. So we started making different designs, and that somehow increased. And it was about talking to customers and meeting them where they were, not where we wanted them to be, or even worse, imagined them to be 

[00:32:18] Janna Bastow: Oh, interesting. And that’s actually often the thing is that the use cases that people are solving for are, you know, there’s often these really interesting niches. Interesting problems that you wouldn’t have in a dropdown in a user research form, right?

You don’t have people typing in going, “Nice wine for date,” for example, right?

… 

[00:32:36] Janna Bastow: I might look up that. But they might, they’re not gonna be often expressing this stuff as they come into your shop, whether that’s digital or physical, until you’re actually there and ask that question.

And the fact that you were there asking that question, that pulled out that information, right? And now you might be able to then position something around that. You might be able to create something or at least test something in that area. But it’s that nature of actually being there and seeing what real [00:33:00] humans are thinking at that point in time that makes sure that there’s new information coming to the system that can be used.

And actually, AI can help with that, right? So AI can be used to help you come up with questions to ask these customers. It can help you come up with ways or brainstorm ways that you might be able to get in front of customers. You know, not everyone has a supermarket they can walk into, right?

Not everyone’s customers are immediately there. But there are often creative ways that you can think of getting in front of customers, and AI can help brainstorm that stuff. It can help you synthesize

What it is that they’re saying or customers who said something similar to this, and three customers who said something similar to this. But it’s often not the one that’s in the room, and it certainly wasn’t the one that said, “Shouldn’t we get out there and go talk to customers?” Because it, AI tends to just say, “Hey, I’ve created something,” and it looks good enough and we go, “Cool.

Ship it. Let’s see if it works.” And that’s expensive. Even though it looks cheap, it’s expensive. 

[00:33:58] David Pereira: [00:34:00] Yes. One of my favorite use of AI, it’s related to this. It’s about a pro- a product experiment. So I talked about naming the assumptions that are critical. You don’t need to create a prototype to test all of them.

What I like is using AI to explore different experiments. I say, “Give me four experiments for each of these assumptions that I can run in four hours. Another four experiments I can run in a day. Another in two days, and another four in a week.” And then we are gonna see loads of experiments for each of the assumptions.

And then what we need to do, we need to read them, understand. And as humans, we are very good with one thing. When we see something, oh, we are good at criticizing and adapting. If we start with a blank page, it’s hard. We need to think and imagine everything. But within this, we see different options there, and then we can decide.

Maybe we pick something and we execute. And AI is good with that if you give the right instructions. [00:35:00] Some experiments can be as simple as, for example, make a post on LinkedIn and c- see how people engage on that, or run a survey and then get results and interview people afterwards, or something like this.

Which connects me to one thing I was talking to Dr. Milan Milanovic. He recently released a book, The Laws of Software Engineering. He said that senior software engineers, they choose to solve problems by often not coding at all, by understanding what is really necessary, and often they are gonna remove code instead of adding because they know what truly matters.

And I’m seeing that some product managers who use AI to choose to identify what makes sense by not creating a prototype at all, but running experiments that will bring you the knowledge that you lack right now. 

[00:35:48] Janna Bastow: Yeah, absolutely. And I think there’s we’re seeing AI being used in two different ways. Melissa actually outlined it.

She said “AI is a thought partner and AI is a speed freak,” are the two extremes that we live in, and PMs are [00:36:00] living that tension, right? And I think the problem is that AI as a speed freak is… You know, it looks great, right? It’s so tempting. Now, there are really great ways we can be using AI to debate with, to help us ask the right questions, to keep us on track with stuff, right?

To help us organize our work, right? If we’re gonna run, if we’re gonna use AI to help us figure out and run 100 experiments, we can actually use these tools to help us keep on top of this stuff, too. Great. But it’s often too easy to just slide into just delivery mode. And I think sometimes the problem isn’t just the product people.

As we know, it’s the people around us who are now questioning the value of product people, because what do we do? We just sit around asking questions and writing specs, and then we throw out the work before we even deliver anything. Why don’t we just deliver stuff? And we see other people on the team looking to replace the product management function with AI, and you can’t replace the product management function with AI because you still need something, [00:37:00] someone doing the critical thinking. Someone being accountable for these decisions and keeping track of all these different sort of aspects.

You know, what have the customer said? What does the business need? What’s been tested? What’s our current constraints where we sit right now? And this is the part that’s at the core of product management, and it’s not something that we’re gonna replace with AI anytime soon.

[00:37:23] David Pereira: No. It is still collaborative game, so many things are gonna happen before you even touch AI. Decisions that happen with executives and so on. So product people will understand how to influence decisions that make sense, and that requires- … a lot of critical thinking. I noticed this last year. It was the moment I created my mastermind, 100X PM, how to move from backlog manager to product lead.

And the idea is to help people sharpen the critical thinking. The reason I created this is because I started seeing everything about AI, and I don’t think [00:38:00] it’s about building everything fast. It’s about becoming a strong product leader, that you have the critical thinking, you can influence others.

And about … being obsolete or not the role, it’s very interesting. Like recently I talked to a product designer from GetYourGuide, and he told me… And he had to play the product manager role for the last six months. Because the product manager quit, and then his team decided to run an experiment and see how life would be without a product manager.

He said, “The beginning was nice. We had more speed and flexibility.” And then he looked at me, “I thought so.” And he said, “Now, I don’t want ever to work without a product manager, because all of the things you have to deal with, all of these people who want everything done by yesterday.” He said, “This is hard work to get people aligned.”

He said- … “Yeah, there are a lot of meetings,” he said, “but the magic happens when you get these people to agree on [00:39:00] something, and then we can move forward.” And then he said, “S- sometimes I felt like I was discussing more the work than doing the work, and I didn’t know how to get people to connect.” And he looked and said, “It…

This is a skill, very important. I do not possess that. AI do not possess that.” And in the team we started begging our leadership, “Bring back a product manager. We can’t live without one anymore.” 

[00:39:28] Janna Bastow: I love that. See, it’s case studies like this that help us really recognize the value and make the case for other companies who might be about to fall in the same trap.

Because I know there are companies out there right now, I’ve worked with some, who are actively trying to decrease their team size, right? They’re trying to replace their sometimes the development and design team, sometimes their product teams as well with AI instead. And they’re gonna end up in a world of pain, right?

Because as you said, you might feel like the first month or so without a product manager, you know, great, we’re delivering a bunch of stuff. But actually it’s not moving any of the [00:40:00] needles that you need to, and it creates this

chaos that simmers at first and then really starts bubbling up, right? This moment where everyone’s then debating as to what it is that they could build. And in a world where anyone can build whatever they want, you’re just gonna end up with a bunch of stuff, not necessarily something that solves the problems.

And I feel like we’ve been through this, right? So I look back on my time with Mind the Product. When we started Mind the Product, the level of conversation was things like, “Am I a project manager or a product manager? What is a product manager?” And we saw these people coming together and realizing what product management was.

And when we started running workshops, it was actually a really interesting data point, because you could see what sort of workshops people were looking to buy. And at first, the early years, this is like 2010 through 2015 sort of thing people were looking for things that were almost like hard skills, right?

So it was like how to do design or how to do analytics as a product person, right? It felt as if product people felt like they needed to pick up the skills that their colleagues had to be to be legit in their eyes, right? Or so that they could pick [00:41:00] up different things and run with it, right?

You saw product people who were trying to get good at writing copy or product people who felt like they still had to code. Back when I started in product, there was this expectation that you came from the coding world. If you couldn’t code, you weren’t a real product manager. Nowadays, like obviously you don’t need to code as a product manager.

And then over time, it started evolving, where people started looking more for soft skills type things, right? You started seeing people moving up into leadership, and they were talking about how to gain alignment, how to build a strategy that people could get behind how to work with tricky stakeholders.

The workshops like that were the ones who started selling. And we could really start seeing that there’s some value being created here, where it was like, the people started recognizing that they were the ones who had to pull together the context and the insights about where the company was and what they needed and that sort of stuff, and keep everybody pointed in the same direction.

And now we’ve swung back around. I feel like we’re into this other cycle where we’ve got really excited. We’re creating all this stuff. There’s gonna be mistakes made. There’s gonna be teams laid off. There’s gonna be product managers who are pushed aside. And then they’re [00:42:00] gonna realize that actually a CEO or, a solopreneur who’s just building stuff and building stuff isn’t likely going to win.

They’re not gonna go faster than team with considered decision-making. And whether we put that considered decision-making on the product person or on somebody else in the team, someone’s gotta have the brain space to do that, and that’s what the product person has done, right? They’re the ones who are figuring out what to do next and what not to do next, so that the company’s more likely to hit their goals.

[00:42:29] David Pereira: Yes, and now it’s a little bit confusing what’s gonna happen. I just read before we started the chat here, Itamar posted something, Itamar Gilad posted something on his LinkedIn about the potential directions, and he put product-centric and technology-centric. So what can happen? And in technology-centric, he said and there is also the output-centric and then the outcome-centric, so the metrics you created. Technology and output, it will be [00:43:00] without PM. AI magic will do everything, and so on, and then we hope for the best. That is the faster version of a feature factory.

[00:43:09] Janna Bastow: Yeah. 

[00:43:10] David Pereira: And then we have outcome-centric PMs understanding what the market needs and creating AI solutions that will help the market. So still looking at technology, using AI to to approach these and so on. And then we have the other part where it’s empowered PMs, like PMs empowered with AI to achieve what we should do faster, but at using AI as an amplifier for that.

And what I think is- PMs now, like you mentioned about project management. 

In Europe I have been in Europe for eight and a half years now, and what I see is it’s very likely that PM, although it stands for product manager, companies will limit to project management. I’m not against project management, but in this case it’s even the [00:44:00] bad version of project management- where PMs cannot make any decision. They need just to follow roadmaps and so on. But what I see right now is PMs just will need to step up to being business savvy, understanding what makes sense to be impact oriented, and also critical thinking. That we need to develop critical thinking so we can make decisions when everyone is trying to use AI.

Because now it is about making decisions that make sense and help the others understand this and so on. And knowing, like, when to use AI and when not to. You talked about workshops. So when I run workshops, you can ask people to run prompts and so on, but I do some experiments. Going back to the assumptions example, I put sometimes one group I say, “We have this challenge here.

Think about assumptions we have behind it.” So they will write down and they will take the [00:45:00] time, like 20 minutes to write down assumptions and so on. Then I say, “After that, you use AI to generate, and then you amplify what you created. You choose what you’re gonna pick from AI,” and so on. And the other group go- Yeah

is I do the contrary. Start with AI and then you add- … what is missing. The second group generally doesn’t add anything. They stick with AI, and they go with AI. They may sharpen something from what AI created. But the first group is very interesting the result. They have identified some things that AI didn’t point out

at all. So they come up with m- I would say a better space to explore what makes sense and not. So the future I see it is more like on the problem space because for now, for years, PMs have been too much in the solution space trying to get out to the problem space, but managing sometimes- We were so close

not always. Yes, we were very close, and now we are pushed back to the solution space. But the [00:46:00] future for me, it’s problem space. 

[00:46:02] Janna Bastow: Yeah, absolutely. And there’s talk often about, is AI gonna take my job? Is AI gonna replace developers, designers, product people? And you know what? If you were a product manager, and we know that there are product managers are, who are doing this ’cause there’s lots of product managers who came in, they got certifications. Free certifications, slapped them on their LinkedIn, and got product jobs ’cause there were lots of product jobs out there.

And a lot of these product managers weren’t actually doing good product management, right? They were writing specs, and they were shuffling things in their backlog, and they were, keeping busy with all the grunt work but not actually doing the critical thinking and owning the outcomes for the business.

And there was a backlash even before AI became a thing. There’s this backlash around we’ve got all these people. We’re not actually hitting our goals, right? We saw some massive layoffs even before AI was really the thing that was replacing them. So there are gonna be product people who are replaced, but they are the people who weren’t really doing the product management work.

Good PMs are the ones who always knew that talking to the customers and understanding the problem space was where there’s real [00:47:00] value. Aligning people and reading clear defensible decisions is a real job, and now the boring parts of their jobs are now being done for them. That’s easy. They’re using tools to go faster with those parts so they can spend more time in that problem space.

There’s going to be a place for product people always in that. There’s always gonna be somebody who needs to be the person who owns those decisions and aligns people and that sort of thing. Whether, long-term we’re called product managers or whatever, I don’t care, right? It’s this skill set that people have that we’re gonna be able to carry forward and think of AI not as a replacement but as a tool that helps us get to those decisions and act on those decisions faster, and that’s gonna be really cool.

And I think we’re getting there, but we’re hitting this point now where I mean, I wish we could amplify our voices here and warn companies of what’s coming, but we’re gonna have to see some companies mess it up, right? They’re gonna lay off the wrong people. They’re gonna build a bunch of stuff, [00:48:00] and they’re not gonna end up Going faster.

And we’ll be able to look back and go, “Okay, so they actually succeed. Here’s what they did.” And so we’re gonna be watching those who really do succeed using AI as augmented AI companies versus the ones who fall flat on their face. And we’ll be able to look back and go, “Ha. You know, of course, if you got rid of all your product people and just replace it with an agent, this is what happened.”

Now, of course, we’ve gotta wait for this to happen. I’m sure there’s people who are listening in right now or in companies where they’re feeling like their jobs are under threat or where they’re trying to make the case for using AI but in the right ways. What sort of advice can we give to them so that they can better communicate what their role is in, in the company?

[00:48:41] David Pereira: I think it’s about making the impact you already have visible. Product people are busy all the time, and then we are so busy trying to achieve something that we forget about sharing what we have achieved. We mentioned at the discovery, like there [00:49:00] is a reason leadership, for example, sometimes despise discovery.

It’s because they don’t see the value. But once you start showing, for example, there is this solution, the cost of delivering this solution would be X thousand dollars and so on. I say, “You know what happened? We just learned it was not needed, so we didn’t do it.” So you can share what your value and help others understand how you’re cr- helping the organization create impact and move the right needles.

So if you are not doing that right now, you can start understanding what is important for the organization right now. It is easy to get busy with a routine and so on, just get the calendar f- filled out and then keep doing things. That will be hard to justify you being there, w- remaining because then you’re busy.

What you need is to figure out how to increase your visibility with impact and so on. So understand what matters for the company, do more of that, share how you are [00:50:00] contributing to that, and also challenge sometimes. If you see things blocking teams from achieving that, why not pointing out and making a suggestion on what you could do differently?

[00:50:13] Janna Bastow: Yeah, those are really good tips. And I think one thing that often helps is being able to better quantify our work or line it up with what the business actually needs. Any tips on how we can line our work up with metrics or things that we should measure to show the return on investment of having product people that helps make sure that we’re making the case for us to stick around?

The thing that worries me that I’m seeing out there are companies or even just individuals who are now talking about how quickly they developed X lines of code, right? And this reminds me of how we were measured at one point in time by how many story points we could burn down, as if that was a good measure.

I feel like we’re back there. So what should we be measuring? 

[00:50:56] David Pereira: Sure. I was just laughing when you said how many source [00:51:00] burn could burn down. Because I was promoted for that once. It’s embarrassing. But yeah, the market rewarded that, as the market’s gonna reward now the number of lines of code. But this is temporary.

That’s not gonna last. So what I see is reflecting on what makes sense for the organization right now. So look at the dashboards you have. Look at the customer journey and try understanding what is going on. For example, in one of the organizations I consulted recently, instead of just jumping in to get hands-on, I looked at what was going on, and there was one thing going on there.

They had a huge number of signups, record all-time. Good. Signups are good if they remain there, if they don’t churn. But then consequently, they also had a huge drop on onboarding. So I asked the product teams, “How might we reduce that? How [00:52:00] might we help customers get onboarded?” And then we started working on that.

And within this, more customers onboarded, more customers upgraded, and that led to more revenue. That’s where you can make the case. So you need to understand the business, like which phase are you? Are you in a growth phase? Are you in a pre-go-to market and so on? What are the metrics that matter? Then look at your team and how you can influence that particular metric.

More lines of code doesn’t mean anything. I will share a very brief story. Once I had a situation that customers were not signing up for for the product at all, and then someone had the idea, of course, no one wants today to create a new account. It makes no sense. So let’s put the social media login there.

We need to put Google social media login. That’s gonna solve all the problem. Again, could do that. We did that. Guess what happened? Nothing. Nothing changed. We remained with the same signup. We interviewed customers, and then customers started saying like- You are asking [00:53:00] for my data before I can even see what you can do for me?

I don’t understand what is in it for me. So I want first to have a look at it, and if I like what I see, then I decide if I want to create an account or not. But I’m not gonna create an account without seeing what you do for me. It was clear they didn’t see the value, so we had to work on our value proposition.

Instead of the trying to convince them, we just let them play around with the product without any login before, and then after that, if they wanted to get more, we would say, “Now, do you want?” That changed, but you have to understand the customers. 

[00:53:36] Janna Bastow: Yeah. Absolutely, and you have to understand their flows as well.

You touched on onboarding there, and I think onboarding is one of these things that often gets overlooked. You know, people

Especially now we’re optimizing for having this, these new features. But actually, is your product capable? Is your product and service as a combination capable of bringing people from needing [00:54:00] to solve this problem or wanting to solve this problem through to actually solving the problem, to understanding how your product is gonna help them solve that problem?

And the nuance is in these flows, and I don’t think AI is good at creating onboarding flows at this point in time, right? It’s good at creating things that they can do once they’re in, but a lot of times we’re not really thinking about these flows from how to get people from identifying the problem to actually having solved the problem.

And whether we call it onboarding or other apps have different ways of bringing people through that flow I think we’re gonna see a lot of companies with really broken flows even though they’ve got impressive looking products, even though they’ve got impressive looking marketing they’re not gonna see the numbers.

And that’s where product people can really start honing in on and saying, “Hey, actually, you know what? I can help us get from these sorts of numbers to these sorts of numbers by really honing in and, figuring out this flow. Figuring how to solve this job to be done.”

So it’s gonna be really interesting I think in the the coming years. And I mean, David, we could have this conversation in three months’ time and reflect back- Yeah … and go, “Oof, we didn’t expect it to go this fast,” or, “Ooh, we didn’t expect it to stall out looking like [00:55:00] this.” So I mean, huge thank you to everybody who’s jumped in on the chat as well.

The chat was just as lively as the conversation with David and I. You know, great to see that everyone’s thinking about this, talking about this. This is a burning conversation ’cause it should be. This is the most interesting thing that’s happened to our space in a very long time. And so we’re gonna keep talking about it like this, right?

You know, wanna hear from everybody else as well. On that note we’re gonna be running another one of these in a few weeks’ time, so keep an eye on your emails. We’ll be posting our next webinars as we go. Today’s session has been recorded and will be shared thank you very much, David, for for taking part.

You’ve also talked about a few references and a few different other resources that we could look at as well, so we’ll post those in a follow-on chat as well. So in the meantime, huge thank you to everybody who’s joined us here today and we look forward to having you back here next time.

David, thank you so much. 

[00:55:52] David Pereira: Thank you. [00:55:53]

Janna Bastow: All right. Take care. Bye everybody, and chat to you again next time. Bye for [00:56:00] now.