Product Management Webinar: Designing at Scale

[00:00:00] Janna Bastow: Welcome everybody. Come on in. A little bit about this series of webinars. So this series of webinars is what we call the Product Experts webinars. We have been running them for quite a while. So we have a whole bunch of them that have been recorded in the past.

We generally do a mixture of either presentations or firesides, like we’re doing today. And we always invite these amazing experts and they always come with these excellent insights that they’ve gleaned from their years of experience. And these webinars are always with a focus on the content.

So today’s going to be about learning and sharing and understanding how different product experts from around the world work. I am super pleased to introduce you to Elizabeth Churchill. I’ve actually known Elizabeth Churchill— and I wanted to look back and figure out when we first met, and Elizabeth, it actually goes back to when you spoke at MTPcon in London 2016, which was actually five years ago today.

[00:00:52] Elizabeth Churchill: Wow! 

[00:00:53] Janna Bastow: So I thought that was great. 

[00:00:57] Elizabeth Churchill: It’s hard to believe it’s been 5 years, as we haven’t actually been able to see each other, like everybody all in our spare rooms. 

[00:01:04] Janna Bastow: Yeah! It’s been a blur, yep. But at that MTPcon London, you did this wonderful talk about storytelling and metaphors in product management and how it applies to our day-to-day—you really brought the human element in for that day, which I loved. I really think it’s recommended to go back and watch that one. 

Elizabeth is a Director of UX at Google and she has a rich background in psychology, artificial intelligence, cognitive science, and she draws on all this to be at the front of creating innovative products over the course of her career.

She’s built research teams at Google, eBay, Yahoo, PARC, FujiXerox, and she holds a PhD from University of Cambridge and honorary doctorates from the University of Stockholm, and for anybody local here in Brighton, at the University of Sussex as well. 

And Elizabeth, I know there’s so much more of your story than that, so why don’t you jump in with an intro, fill us with any of the gaps and tell us about your background and how you got to where you are. 

[00:02:02] Elizabeth Churchill: Yeah. Thank you. Thank you for that very gracious intro. Hello everybody. When I started off as a psychologist and then got interested in technology and human factors, really, and technology sort of, you know, in the 1990s, late 1990s, I started to really think about collaboration tools and how we use these sorts of technologies to actually communicate and collaborate.

And so, seeing where we’ve gotten to now is amazing. And it’s actually, I was joking about the last year and a half, the innovation we’ve seen in the last year and a half for these collaboration technologies that allow us to work together at distance has been amazing. So I’ve always been very interested in how people work together and how they work together when they’re in the same space, as well as when their distributed.

So Janna mentioned a company called FujiXerox. And one of the projects I did there was to look at how researchers and engineers could collaborate between Tokyo and the Bay Area and looking at all different kinds of tools to see how we could really support innovation when people are at a distance. But not just at a distance geographically.

Also, with sort of cultural distance and language difference. So thinking about how will these tools like the tool you just saw, really help us create innovative new products, create great new research streams. That’s been sort of like a passion for me, and that has sort of continued in all of the different roles that I’ve had.

So now I’m heading up a research and UX design group for a new operating system called Fuchsia that got launched a few months ago from Google. It’s out there, so go and have a look. And it’s very early days, very early days. But, when I met Janna and the team, I was also running a research group for Material Design and working with some fantastic researchers on Flutter.

So these are all production tools. These are all tools that people use to create the kinds of products and experiences that we’re all very familiar with. So that’s the sort of space that I’ve been working in. How do you take a deep understanding of human problem framing, innovation, collaboration, production, prototyping—how do we take all of the understandings of those things and build the tools that allow you to do great work? Just like the tool that we saw from Janna and team, how do we provide the tools people need to do the great work that they need to do to create the consumer devices? So, I like to say consumer grade designer and developer tooling is kind of what I want, so that designers and developers don’t end up having terrible tools—arcane tools, tools that don’t integrate—because I think if we give them the great tools, if we have great tools, we can focus on the creativity and understanding how the product lands, and doing great research to make sure we’re offering high quality consumer experiences.

So that’s a sort of, bit of a ramble, but that’s where I’m at. 

[00:05:15] Janna Bastow: And that’s fascinating. Given the work that you did on Material Design, I’m super excited to see what comes out of the work that you’re doing on this new Fuchsia thing. I don’t think many people actually know much about it.

Could you give us the highlights as to what to expect or how to frame it?

[00:05:33] Elizabeth Churchill: It’s very, very early days. So Fuchsia is open source it’s out there. It’s a new operating system. And so it’s a very exploratory space. And we have launched Fuchsia on a few consumer devices and we’re seeing how they behave in the field, if you like. It’s just a sort of exploration into what an operating system that builds on and collaborates very effectively with existing operates operating systems—what are the innovations in that particular space? So it’s very early days and it’s very much a sort of exploration in how we can take the best of what we know about rating systems and enhance that.

So the tooling work that I’m doing specifically is just to look at the tools that the developers of the operating system are using themselves. So it’s very much that sort of meta analysis. So, there’s not a lot more to tell you other than that, but please go and have a look at the news reports. 

[00:06:28] Janna Bastow: Excellent. It’s a really exciting to see this space being explored. Definitely a space to watch. 

[00:06:33] Elizabeth Churchill: Yeah. And I mean that said, you know, there’s always innovations within the existing operating systems that are out there. There’s a very active work going on within this whole space. It’s just that the sort of, what’s the new twist? What’s the next thing?

And we’re exploring with our collaborators. So there’s always innovation going on, but it’s under the hood. So as consumers, we often don’t see it.

[00:06:53] Janna Bastow: Yeah. And it’s an opportunity to rethink a lot of the assumptions that were made about operating systems and how consumers and how developers work with them. These assumptions would have been made years ago and probably don’t necessarily hold true anymore.

And so this ability to start again from fresh is a, it must be freeing. 

[00:07:10] Elizabeth Churchill: Yeah. Yeah. And you know, it’s still building on the best of what already exists. And as I was saying, deep collaboration, so this is, you know, in no way something completely new, but it’s just about where can we build some innovation.

[00:07:22] Janna Bastow: Yeah, absolutely. So, you’ve got undoubtedly a unique role in your position at Google, compared to most of us, you have a huge team and of course, a huge user base to draw on. And I think there’s so much that we can learn from you about what things look like as they scale.

And I’m particularly interested in the research and innovation aspects. So what’s that like in your world? 

[00:07:41] Elizabeth Churchill: Well, I mean, I think that’s interesting because our current userbase in my current role is not big.

And it’s mostly internal, but, as that goes forward, we will have more externally, but more generally, if I think about something like, material design, which has a big user base in that space, scaling was very interesting, because one of the things that we did with research there, it was very early set up a sort of analytics program.

So one of the things we did was to say,”Hey, you know, what is the sentiment out there?” So what are people saying about Material Design? And so one of the members of the team, when we were building that, Jude Yew, he built this program where he prototyped a lot of going out and looking at Twitter. He had a look at sort of Git, and like comments there, he basically went out and got data sources, around all of these different ways in which Material was showing up externally. And then we would take that information in and we would start to look at how Material was landing with our external developers internally, we would do interviews and we would have panels to find out a little bit more about how Material was being used internally.

In addition to that, we were doing a bunch of research on the specifics of the components and whether they were usable at all. So that, the scale there was to use Amazon Mechanical Turk and put experiments out and have people try out components and a chap called Michael Gilbert drove that program. So we scaled research by using existing platforms internally and externally to get feedback on the quality of the product, the components, et cetera, the quality of the guidelines and how they were landing, and then we would also look at the sentiment externally and how things were landing, and internally through interviews. We also had a program called happiness tracking, which is like a little pop-up, which would ask you how you’re doing on the external site. And Abla Hamilton ran that program and continues to. So we scaled our understanding of the design system through these mechanisms and through surveys and so forth.

Then brought those learnings back in, worked with the amazing design team and the writers to create the content, and just have this continual—I’m sure you’ll all recognize this— iterative evaluation of how Material Design was performing in different contexts. And then the promotion. There’s a fantastic team who did a lot of promotion through conferences, writing the Google design blog.|

Then you start to sort of scale out the influence, but always in tandem with this evaluation process to see how things are landing. So, interestingly, I mean, I think that’s very much like launching any kind of product and so adapting a multi-method approach to understanding what was going on and knowing which method to use.

And back to that thing about the problem framing of what are the questions we need to ask. That was something that we just routinely did. And so when people say to me, how is it different working on a product or a platform like a design system, or like working with developers? There are things that are the same.

There are things that are different from doing consumer experiences. So we didn’t really have that ability of, you know, hundreds and thousands of users are going through a particular workflow and it’s the same workflow, the same critical user journey say onboarding or paying or whatever. But what we did have was audience segments and critical user journey—CJs—that would be for different kinds of design or different kinds of developer.

And the term that Jude has coined for this is constituencies. So you can think of them as segments, but by thinking of them as constituencies, we’re starting to really think of them as, you know, groups of people who have different needs, who use the tools in slightly different ways and who are sort of learning and growing, and changing. So very sort of dynamic space, because we’re also sort fo wanting to get people going from, you know, potentially using a little bit of Material Design to a lot more, growing that engagement. So, you know, I think you would probably see, there are some things that are similar and there’s just some slight differences when you’re doing this platform kind of approach.

[00:12:09] Janna Bastow: Absolutely. And I love that you shared some of your actual tactics —how you scaled some of your testing methods and whatnot. And one that stood out to me. Cause I hadn’t always thought about this is something that you could use for this, which was a Mechanical Turk and I’m not even sure everyone actually knows what Mechanical Turk is. To get everyone up to speed, do you want to give a little bit of background as to what that is and how it worked? 

[00:12:29] Elizabeth Churchill: Yeah, so it’s a platform. Basically you essentially crowdsource participants to come and do your tests. Basically what we did was we would create prototypes, which were sort of clickable prototypes— and Michael Gilbert was the genius behind a lot of this— and then you go out and you recruit people and you pay them a certain amount to go through a certain set of tasks. So it’s very much like other services that are out there where you can actually go and have a company do this for you.

But we were just doing this in-house because we had these components that we were building, trying out. And, I think there’s a couple of talks out there where Michael’s work on text fields in particular, where we, prototype this technique, was very, very influential on what we did with the texts fields in Material Design.

So basically you can use the platform to get hundreds of thousands of users to come in, or a few, if you want and do the tests. And so there’s scale again, you have a lot of people trying something out that’s innovative, that’s new, that isn’t completely robust. It’s, it’s a prototype. We worked with some of the brilliant designers to sort of think about what all of the parameters of the text field that you could push. And then we came up with all of the conditions where somebody would go through a different flow as a participant, and there was something like 144 different conditions that you could go through, if you looked at the weight of this text field box or the, where the label was placed, and we had them go through these flows and then you can see where people clicked and then you can do a little qualitative evaluation of how they felt about something. So, it’s just a very powerful platform.

If you have a prototype and if you want to do something in house, because you don’t have a prototype, you can share externally with a company that does this. That kind of scaling was really helpful because you can very quickly get data around whether something is working or not. And for whom it’s working.

And interestingly, on that study, one of the things that we found was that, we’re not telling the designers and the design system team which texts field to use. What you do, what we were doing was laying out a space, because we had enough data points, of which ones work better than others and on what kinds of dimensions.

So back to the sort of square box of the text field, we could see where people were clicking under the different conditions. So imagine there’s all these different, like thick bounding boxes, just a slim gray line, all of the different things that we can push to what piece of text field, and we could see where people were clicking.

And so we would come and say, well, the thin line people are clicking in various different places. So it’s a bit like doing eye tracking, you know, where the tap is, but if you have this kind of text field or this, within this space, these ones are where we actually hypothesized and wanted people to be clicking.

So the platform allows you to have all of these different versions and then you can sort of try it out. Does that make sense? 

Was that helpful? 

[00:15:25] Janna Bastow: Yeah, that absolutely makes sense. And like you said, there’s services out there that will offer this sort of thing, like an all in one, but you can roll it your own and do it both at a small scale as well as huge scale as you mentioned.

And I guess as you scale, it becomes much more cost-effective if you create a system by which you can roll your own. And, it sounds like you’ve found a way to do that. Brilliant. I love that. 

[00:15:45] Elizabeth Churchill: Somebody was asking about the IP. And, I think that’s a really good question. I think within Material Design, because it’s external and open and we have a community, IP does not come into play as much.

Now that’s different from saying that we don’t have our own— some versions of Google specific branded aspects of Material that internal teams use, but it’s not available externally, but that’s not necessarily IP in the same way. That’s more like sort of brand identity. In the other space, again, we’re open source, so the notion of IP specifically doesn’t come up in my life very much. It might, may well do within the engineering space, but in the UX space, it really hasn’t a lot. You can have a site, fuchsia.dev, and you’ll see the tech writers, a couple of writers on the team, who’ve done a really nice job of doing an explanation, but that’s not IP because we’re actually out there in the world.

But it’s a very good question. I’m sure the engineering teams would have a very different answer for you for that. I’m not— I don’t have a lot of insight into that. 

[00:16:50] Janna Bastow: Yeah, that’s fair. And thanks for tackling that question. And if anybody else has questions, drop them in the chat, or even better in the Q&A section, and that way we can easily spot them. 

Another thing that I’d be really curious to hear your point of view on is, your users are developer users, which might be unique compared to the type of users that I usually work with, which are B2B type users for me, or, direct consumer type users. How did developer users differ? What’s unique about them and what’s the same? 

[00:17:20] Elizabeth Churchill: Well, I think that’s such a fascinating question. Basically, I’ll probably offend all the developers on it. So I think in terms of consumer, a lot of the—and then I’ll go to the B2B—but in terms of consumer, I think a lot of what we use as consumers, the goal—well, let’s say there’s discretionary use consumer, which is more like me, you know, going to my favorite app, for reading the news or watching movies or something. I think for that sort of discretionary use, we want to keep things pretty simple. We want to make sure that we’ve got a wide range of, say, expertise, for want of a better word, and we want to make sure that things are pretty obvious, that we’re using really great calls to action, that we’re clear in our explanations, and that we are trying to be as accessible as possible. Where, accessible, here, I’m really thinking about interpretable and understandable. And so findability is really important.

As you go through some B2B or through productivity apps—productivity apps require a little bit more expertise, for example, if I’ve got a spreadsheet app or something, a little bit more expertise is needed. So if we’re going from discretionary quick drop in, look at your favorite photos, then we’ve got some productivity apps, then we’ve got really complex experiences like the B2B. So for example, if you’ve got some kind of management system for some kind of account, you’ve got expertise there. You’ve also, often now, you’re moving into, needing better security, you’re needing, moving into needing better robustness and reliability.

You really don’t want your relationship management thing to go down. You really want to be careful about making sure that whatever the services you’re delivering is absolutely reliable. So you’ve got expertise, reliability, et cetera. Then you move into the developer space, and you start to really get much more in to, again, very, very expert users and the tools. Sometimes the tools, people enjoy having really complex tools because it makes them feel like they really, you know, that their expertise is being shown off. 

You’re starting to get into designing tools where the knowledge that is needed may be in multiple spaces and may be very complex. I’ll give you a concrete example.

You’re in a development environment. Where do we design the documentation? Does the documentation get linked into the development environment or doesn’t it? If it does, then are we producing the documentation or are we reproducing something that is externally available? What are the everyday practices that the developers are using?

What are we changing such that we completely change the nature of their practice or their role, and it’s actually super disruptive, versus—cause they need to keep producing. They also need to do a lot of testing. So developers are creating a lot of artifacts, which need to be tested and debunked. So now you’ve got a subset of tools and testing tools, debugging tools.

So the kind of constellation of all of the different things that need to get used and whether people are in the development environment or popping out or going to the web and looking something up or copy paste coding, and then it doesn’t work in your context. It’s a very, very different problem solving context where the moving parts, and the information that’s needed in order to be successful are multiple. So each piece of that might look like the discretionary user app, but the whole ends up being a lot. 

[00:21:11] Janna Bastow: So essentially you’re building for the ultimate super user type of a user, right? They’ve got to be able to understand and deeply manipulate the product, to be able to do whatever it is, express whatever it is that they’re trying to do, in a way that also pleases and serves the needs of their end users, who sometimes are just trying to check their bank balance or find a nice photo or do something simple with it.

[00:21:34] Elizabeth Churchill: I love that term. Super-user yeah. And I think what’s also interesting, is that in that sort of development environment world, I’m saying “developers”. But I mentioned Jude saying before constituencies, because, if you’re looking at an operating system, a kernel developer is very different from a UI framework development.

They have overlaps. They have overlaps in knowledge and interest and expertise and tool usage. But the problem solving thing is often different in that the resources that they are going to have different resources, right. It’sa really rich place, it’s fascinating. 

[00:22:07] Janna Bastow: Right, yeah. And I guess that takes me to a question that somebody asked, and they said, who is the main user for Material Design?

And I suppose I’ll add to that, is that different for Material Design versus Fuchsia?

[00:22:19] Elizabeth Churchill: I love that question because when I first joined Material Design and aesthetic research Material Design several years ago, and as I say, I’m not running that anymore. They’ve got a fantastic team now, being headed up by somebody else. I still chat to them regularly. 

When I first joined, there was this idea of the visual design, graphical design, interaction, design user. And as we evolved and some research that we did at— a woman called Sarah Cambridge did with, Abla, who I mentioned earlier, Abla Hamilton, we actually went out and looked at who was using Material and it turned out that more and more developers were using Material. And so we ended up with this category called the hybrid, and that was this sort of emergent category at the time, several years ago now of designers with excellent design chops who are also programming, who were also developing.

So there were these sort of hybrids who have some programming chops who want to actually work with code, who want to take the amazing visual designs into code. And back to the thing about tooling, we now have so much better tooling for being able to do that. To cross what was a chasm from, you know, here’s my visual expression, I hand over to you and it’s like a mock, and then you implement it. And then we go back and forth and try and get it to where my vision was for it. These hybrid folks are people who have great visual design, great interaction design, and they want to code. So, over the period that I was working with Material and much, much more now they’ve really invested a great deal in providing that code. And so if you look at the evolution of the external site, which is material.io, you’ll see, we started to really get more code out there, more code snippet, examples, and also try and have the navigation of the site where we understand, based on your entry point, where you want to go and what you want to do.

So the evolution over the time I was working was so exciting, because we really did start to kind of understand this hybridity and understand that what we want to do is give people the ability to stay purely visual if you want, work on interaction, and/or code, if you want. So that was a beautiful evolution.

So the question here, a UX/UI, or developer, or all of them: All of them,

Anybody who’s interested in creating that visual interaction experience. And of course, the other thing was that, when I first started, Material was originally very much designed for consumer apps and mobile. And as we evolved over time, we found that enterprise users were really interested in Material and desktop, and, it became a much richer set of recommendations and back to that, things like people with a lot of expertise. So information density became a research program to look at for different users—super users—with different kinds of apps, productivity, information density might be different than for discretion where you use the app, which you don’t want sort of white space, and you put beautiful images that you can see and the calls to action are very much fewer. 

[00:25:41] Janna Bastow: Right. Yeah, that makes a lot of sense. And you know, I’ve often heard people talk about how you shouldn’t create a product that’s targeted at everyone. But at the same time, sometimes you do, when you’re talking about transformative products that are going to have this broad user base, sometimes they do target such a wide base, which is fine.

As long as you deeply understand the different types of users that are going to be using them, which sounds like you and your team very much have a handle on and the different types of use cases that they’re going to be going down and how you might change things like the information density, depending on what flow or what kind of job they’re trying to achieve.

And that sort of thing. 

Yeah. 

[00:26:20] Elizabeth Churchill: Yeah, no, absolutely. And what’s so fascinating, there’s some great work that’s been done, and this is going to just sound like, you know, everybody’s going to go. Oh yeah. Yeah. Because you know, one of the lovely things about watching people use whatever you’ve created is the hacks and the work around specific interesting things.

So here you are, you’re sitting there going all right. Well, this is what I expect my user to do. This is what most people are going to do. And then suddenly you observed somebody doing something and you’re like, it would never have occurred to me. That’s a hack, it’s a workaround. I’ll give you a really kind of banal example, but sometimes, you know, there’s this sort of psychological concept of functional fixedness.

So you have this thing and you think it’s just, you know, that’s what it’s used for. Imagine you’ve got a pile of papers and it’s very windy and you’ve got a hammer. You put the hammer on the paper, it’s suddenly a paperweight not hammer, right?

So, you’ve got this functional fixedness. You never think that the hammer might be a paperweight. Anyway. So the example I was going to give you a sense that the airport’s a couple of weeks ago, and you know, you get your little ticket and they say, write down on the ticket so you can see where you are, which floor you are in which segment you are, because otherwise you come back—I don’t know if you have done this, you know, where did I park my car? I have no idea—and so they say, write it down. Okay. 

A sort of hack, it’s not really a hack, but you know, it’s to just take a photo of where you are. Right. And so, I just did it automatically. And I thought, wow, a few years ago, I was still writing things down and I’m sure that the camera—camera can capture anything—but the camera as a navigation device to find your car is a sort of just a slight shift of how you think about what the camera is for, right? And I think, when you have a product and you see somebody using a product in an unusual way, you’re like, oh, that’s a real opportunity for some kind of innovation. So back to your point, there’s a whole segment of users out that I didn’t even know about who I could actually sell hammers to as paperweights. 

[00:28:26] Janna Bastow: No, that’s brilliant to hear. I mean, that’s one of my favorite things to do, is see how people are trying to break the product, right? What are they doing here? People come to me kind of apologetically going, well, you’re not going to like how I’ve set this up, but here’s what I’ve done.

I’m like, no, no, show me, this is fascinating because you know, we’ve learned so much from seeing how product people change the way that they work and we’ve adjusted the product around that based on what we’ve learned from other people. It’s great to watch and understand, and take that back seat as well. Let them guide and show you what they’ve done, and never make them feel like they should apologize for what they’ve done, because what they’re doing is they’re solving a problem with the tools that they have. And sometimes what they have is a hammer and that hammer is just going to be the paperweight and that’s fine.

People will innovate, they will find innovative ways to use things. And sometimes they’ll use your tools and show you interesting ways that you can solve problems or even new ways of finding value. 

[00:29:20] Elizabeth Churchill: Yeah. Yeah. You know, one of the things I love is that people have told me that, you know, developers working with developers, they were going to be like, very opinionated and not be interested in what other people do because everyone’s a special snowflake.

And that is so not true. My team has been working with these amazing developers when we show them how other developers are using the tools that they’ve created for other developers, they are fascinated. And so, you know, that “will my user like what I’ve built” it’s, I think it’s not everybody, but more people have that feeling than don’t actually.

And so, you know, I just found it lots of joy working with developers because they’re fascinated by design and research and how what they’re building can be better for others, and it’s a fantastic partnership. And I did not expect that because I thought, oh, they’re going to be grumpy with me saying that what they built isn’t working, but not, it’s really, really, really great.

[00:30:17] Janna Bastow: Oh, that’s great. That’s wonderful. And so, what does it take to work in developer tooling? Does somebody need to have a development background themselves to work with developers and to work in this sort of space? 

[00:30:27] Elizabeth Churchill: I mean, I think it helps. I think it helps because you have a different kind of empathy, but, I think if you, don’t, what you’re bringing is fresh eyes and novelty to something.

And so you’re looking at the complexity and saying, you can reach more people by being less complex. I’ll give you another example. The external site that we have. We have a couple of tech writers who have been really working on making the language plain. And I think that invites developers as well as people who are not developers, and their plain language has been really a partnership and saying, what is it you mean when you say this?

And again, back to that point I just made, if you say to somebody look, “if we make this really simple, more people are going to use it”. And then I think that’s a lovely feeling for people, but if you have a development background, you might miss some of those things because it’s familiar to you.

Right. So, if you say to somebody, how do you use this particular tool? And they’ve used it all their lives and they say, oh yeah, you just go in and you do that CTRL, hash, whatever. And they’ve got all the shortcuts. But if you ask me, I would go like, well, why do you do that? What does, what does that mean?

And I think we underestimate how inviting it is when something is simple. And we underestimate how a naive eye can actually really turn something around and make it better for everybody. And so having a development background, I think is helpful, for the empathy, but it can stand in the way of seeing a problem.

And if you don’t have a development background, you can often feel like a stranger, and out of your depth and overwhelmed, and it’s really important to not have that make you feel negative about yourself. So I don’t have a development background. I mean, I have done coding in various ways.

I’ve certainly, built websites and, back in the day I was, doing sort of AI—little AI systems, I’ve done analytics, but I’ve never built a system. I’m not software developer by any stretch of the imagination, but I find it absolutely fascinating. And so, bringing that lens of being really curious about humans and people and what the tools are that they need, that’s the most important passion and skill to bring.

[00:32:55] Janna Bastow: So essentially like any sort of skill you got to be passionate and try to help them solve the same sort of problems that they have, right? You’ve got to be interested in helping them meet their goals, which doesn’t require you to have the same problems that they have, just to be able to empathize with them.

I get that question a lot is, do you need to have a development background to work in product management, which now, I think everyone knows that nowadays you don’t need to be a developer to be a product manager, but it does help to have what I call a sense of humor in line with the development process, to understand that sometimes, something that looks simple is not straightforward, to understand, that sometimes things are complex, and to be able to bounce something off a developer and have a good conversations with them, not necessarily about the complexity, but just to be able to empathize with them and work with them to map out and talk tech about what’s going on in this space.

And if you ask them to explain something, right, map it out, put it on a whiteboard, any good person would. And, you can understand so much if you break things down into smaller pieces, which is no different than if you were working with, I don’t know, vets, like you don’t know how to work on animals, but that doesn’t stop you from being a product manager for vet for example.

[00:34:11] Elizabeth Churchill: Right. Exactly. If someone was asking you, what advice would you give a user research to speak more in the language of business outcomes, work with your PMs product managers. Be really, I mean, that’s such a great question because, you know, there’s that classic exercise to do that, you know, the many hats where, you’ve got your research hat on, your PM has got the product hat on, and the PM is really thinking a lot about the business and the landing and what are the measures and metrics.

And so you’ve got so much to offer each other because you’re both basically researchers trying to think about what the data is that is needed in order to—back to your point a minute ago—get the goals achieved. As a researcher, your goal is to deeply understand the product in use. So, it’s not as much about the user as in use.

And then you can come and say to your PM, Hey, I know we’ve got this measure here, which might be daily active use. It’s not the right measure because actually what we should be measuring is whatever it is you’ve come up with. And then you can start to say, if daily active user was our measure that we were thinking about in terms of audience and scaling and growth, which comes back to the business.

Oh, maybe isn’t that, it’s actually, whatever it is that you’ve come up with, you know, maybe it’s not daily active use of the actual product. Maybe it’s the click-through rate on one particular aspect of the product. So always be in dialogue with your PMs and your PPM and PM partners, because they are looking at the product through a particular lens.

You are looking at the product through its use lens, and those two things need to come together and be in constant dialogue. And I think it’s really important to have these different hats and to know that you might well be under competing incentives, but it is everybody’s job to understand that the incentives are understood. Understood to be potentially competing and then understood how alignment comes about, and how you do testing in the field to figure out if there are competing incentives and competing values, which one do you want to really invest in? So, that comes back to the storytelling talk that I did ages ago, five years ago, which is you’ve got different storytelling skills and it’s almost like having different genres of the story.

And what’s really important is when those two stories start to come, not in conflict, but be evaluated to understand which is the alignment that you need. So that’s what I would recommend. 

[00:36:49] Janna Bastow: Yeah, absolutely. And I love that you’re speaking to alignment and to add to that, I would also recommend making sure that you’ve got that alignment across the business as well.

And this actually echoes something that we talked about— we had Joe Leech on the webinar with us a couple of weeks ago—and he pointed out, he’s like, well, a lot of these companies I talked to, they can’t get their outcomes aligned because their business strategy isn’t set.

And so you need to take it up a level, to the bosses, to the execs, and figure out, what is it that we’re actually trying to achieve before we start breaking it down into these metrics. Because ultimately if we end up, trying to just, you run off and do this metric and I run off and do this metric, they’re going to end up overlapping or clashing or pulling in different directions if we don’t have that alignment from above 

[00:37:35] Elizabeth Churchill: Such a great point, it’s such a great point. And what’s shocking to me is, you can have a very small team and not be aligned because people aren’t actually talking. And so you might think of this as a giant company problem, because it’s like, you just get lost in the big company, but actually people aren’t talking and aren’t being honest and aren’t having those very polite but crucial conversations, which can be deeply uncomfortable if you don’t have a space of psychological safety to actually uncover those things, and you don’t have an inquiring mind to be able to ask and probe that functional fixedness I was talking about. That is, you know, people get into their tracks and they just, you know, they don’t see that they are making all kinds of assumptions. You know, having that to really push on those things. We’ve been doing these things called Assumptions Workshops.

So there’s a woman on my team, Paige Pritchard, and she has been running these Assumptions Workshops with some of our developers. And she just says, well, why do you think that? What do you think the user is going to do? Why do you think that? And just in a very gentle way, in a very collaborative way, but I love that unpacking, but you know, creating the psychological safe space where people can probe like that, I think is so important.

[00:38:55] Janna Bastow: Yep. And it was actually at Google where those early tests came out where they determined the factors that led to the most successful teams, which were the factors that go into psychological safety. Are people in teams willing and able to speak up and talk about the assumptions that they’re making or point out if they think somebody else has made an assumption or made a mistake, or if they’ve made a mistake. These are just critical factors, but they’re tiny little cuts that you don’t really notice. The difference between somebody saying, I thought about something, but I didn’t speak up about it versus somebody going, I’m just going to call BS on that and just say something, is huge in the long run for companies, both large and small. And so I love the concept of these Assumptions Workshops. Are these something that are run on like a regular cadence, like a retro, or are these run like ad hoc?

How do these work? 

[00:39:47] Elizabeth Churchill: I think right now, there’s a program of them, but not that frequent because we’re very short staffed and you know, all the rest of it, but usually what she’s been doing is running them when there’s a big decision about to be made. So are we going to, where are we going to invest in the tooling? What do we think the readers of the documentation already know? What are the assumptions you’re making about who those readers are and what they know? So where there becomes a sort of open question around making a decision on pushing forward on something that’s been designed, whether it’s documentation or tooling, just saying all the people in the room, who do you think you are designing for? What are the assumptions that are behind this? 

One of the ways that I want to do more of this is to make assumptions around accessibility. So back to accessibility, but a different kind here, there’s the knowledge base accessibility, but, in terms of, readability, vision impairment, in terms of— what are the assumptions you’re making around the capabilities of the person who’s going to use, whatever it is that you’re creating, and those capabilities might be situational. We often talk about, vision impairment, as being of the user, of the person. But when you think about, trying to look at your phone while you’re in the car, where there are, there are things that you can’t do that, which is why, you know, you talk your texts or whatever—and I don’t do that either.

So I want to do a lot more around that. So that’s sort of an area I have some fantastic people on the team who were very interested in this area. So I really would like to start to unpack the assumptions that we’re making about the context of use rather than the user. 

[00:41:35] Janna Bastow: Yeah. I love that. And you know what, it’s never too early or too late to just lay out assumptions and start unpacking them, right? There’s always something that you can learn from it. And there’s never, there should never be shame in letting out even the most baseline assumption, because you never know who’s not actually on board with it or caught up with it. 

[00:41:52] Elizabeth Churchill: So back to that thing about, you know, do you need a development background?

It’s very easy if you’re not in a safe environment to think that because I’m not a developer I’m not as smart as you, and then you start getting into that. And that’s where an assumption comes in, which is you need to be as smart in the domain as the other person. And that’s an assumption that I have to take away, right.

Because I’m just as smart. We’re just not at different things. It takes a village of all different perspectives. So I think these assumptions can come in, not just when you’re making assumptions about, you know, the thing that you’re building and what you’re inscribing to, what you build, but it can also be what assumptions am I making about whether I belong or not, or whether that other person feels like I belong or not. And I think that’s where the silencing and stuff can occur. 

[00:42:40] Janna Bastow: Yeah. Yeah. That makes sense. And so when you’re building such a big complex product, how do you deal with all the competing needs? I mean, from solving basic problems that it needs to solve through to accessibility and everything else in between.

[00:42:56] Elizabeth Churchill: I think that’s a good question. And I don’t have a really satisfying answer to it. I think there are things which are just table stakes, right? So you have basic standards around accessibility, basic standards around internationalization, basic standards around, code quality, basic standards are around how much tech debt can you incur before you need to actually address it. And then, so there are table stakes and then there’s that sort of gray area of, are we going to just do the table stakes around this or are we actually going to really be excellent and innovate in this space? And that’s where your prioritizations come in, in conversation with all of your tech leads with your PMs.

And my team certainly has done a lot of research internally on our own processes around that to, say what processes work and what does. And how are we doing those prioritizations? And so that’s not a very satisfying answer, but I think you’ve got your basics, your table stakes, and everyone can agree upon those, or should, because there are basic standards, right?

And then there’s a sort of area of, all right, now we have a whole bunch of things we need to, we may be understaffed under resourced in certain areas. What is the long-term roadmap? What is the near term roadmap to get us to that point? Where are we going to invest? And I think every team has that and you can be a giant team at Google or a small team at a startup.

And those again, crucial conversations, and alignments need to be in place. You mentioned OKRs is when you were showing us the tool that you’ve built, I love that because what you’ve got that is like, where does this fit with our OKRs? Because you’re bringing that together, these different mechanisms, these different strategies and tactics, these different languages of prioritization. And so really saying, where are we going? What is the ultimate objective? What are the sub objectives and the key results towards that ultimate objective is sort of like Russian dolls, right. And so the fact that you’re showing the OKRs in that tool is so important because you want to make sure that those things are in conversation. 

[00:45:07] Janna Bastow: Well, yeah, that’s exactly why I love OKRs or any framework that’s in that sort of vein, which is essentially the idea that you’ve got the ultimate business objectives, right, what it is that you’re trying to achieve, and that should all be pointing you towards that vision, right? The ultimate company’s vision, what it is that it’s trying to be, and everything that everyone does within the business should be able to be justified by some facet of that. And it should all stack back up and you should be able to take your OKRs as in play them down and see that, okay, this is what everyone’s doing and you should be able to play them back up.

So if everyone does all the things, it should realistically stack up to more or less what we said we were going to do. 

[00:45:44] Elizabeth Churchill: Yeah, that’s one of those crucial conversations. 

There’s a question here. What would you recommend for a team without a UX team, when the emphasis on what to build is driven by what clients say—say they want to avoid faster horses—we have no analytics, sadly, and then there’s a longer question, which I’d like to look at as well. So I think this question is a great question, but you know, there are so many amazing resources out there.

So you might not have a UX team, but please, the problem framing that you can come—if you can get your clients to frame the problems slightly differently, and if you can look at what resources there are that they can turn to, it becomes a kind of educational consultancy. Now you may have clients who are completely resistant, and so this is down to your persuasive powers, really, but there are so many resources out there. And so I think about how you can help them with the problem, or the question in such a way that I either they ask for the UX, or you can make recommendations from resources externally that they may want to look into.

But it’s a very tricky situation and it really depends on who those clients are and how intransigent they are. So yeah, that’s—I’m sorry, you’re in that situation. It can be very frustrating. 

And then there’s a unique problem: users have been using the same product for 10, 15 years to the point where it’s by muscle memory. Yes, indeed! A new product has no reviews based on newcomers and working remotely. Yeah. I can see that. How do you reduce the friction and ease the acceptance? I think that’s such a, that’s a hard question. I think what one might be able to do is have, if there’s a way to kind of separate out a little bit, into what essentially it looks like two products, not quite, but you know, and so by showing the value to a few people and, advertising the use cases and the successes and the testimonials, and really doing some, using some platforms to talk about why this is better in terms of a work practice. So you can showcase how the change in practice is actually a value to the existing user base.

If there’s any way that you can do that. And it might be, you know, creating a small panel of existing users where you’re also sort of introducing them to the new, hand-holding a little bit. So I think just changing will lead to this disruption. But if you think you’re going in the right direction, use any mechanisms that you have, whether it’s written, video showcasing, testimonials, hand-holding people into the new practice to show them the value. It just might take a bit of time. 

The one thing I would say don’t do, one of the companies I worked for, which shall remain nameless, rolled out a new product when it was new version of an existing product and had exactly what you’re talking about. And the existing user base was very vocal, but also, frankly, engagement was quite down. So what they were using the product for was minimal. There was no growth but they were very vocal, whereas there’s this new opportunity for a completely new audience, which was where the engagement was, through testing, much, much, much higher and likely to sustain more kind of loyalty and value across many more of the features. Anyways, so the team there, because of the vocal nature, rolled back to the original version and that product is now completely gone. So don’t do that. 

[00:49:26] Janna Bastow: So there’s a lesson there. Absolutely. Excellent. Well, thank you so much for handling those questions. And, thank you so much for taking the time to chat with us today.

I realize that there’s a few questions that have come in very last minute here, but we won’t have time to get to them today. But, Elizabeth, thank you so much for taking the time to join us today. You’ve been absolutely wonderful to chat with. For everybody, thank you so much for joining here today.

The talk here today has been recorded and it is going to be up on our YouTube channel. You will be able to see a copy of the video and the transcript up on our sites. Keep an eye out for that next week, and the next webinar, our next Product Expert webinar is going to be about Embracing Agility in an Organization Built for Predictability.

And that’s going to be on October 14th. It’s a Thursday, same time, same place. And that’s going to be with Emily Tate, who’s the Managing Director of Mind the Product. So see you all back here again, come again, bring your questions, bring your chat. And I look forward to having you back here again. And Elizabeth, once again, thank you so much for joining us. It’s good to have you all here.

[00:50:30] Elizabeth Churchill: Thank you for the great questions, everybody!

[00:50:32] Janna Bastow: Wonderful! Thank you. Goodbye.