Lessons from the Dot-Com Bubble: How Product Leaders Should Think About AI

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: In the immortal words of Bob Dylan, ‘the times they are a’changin,’ and change on this scale is… well, it’s a lot of things. It’s scary. It’s exciting. It’s unpredictable. And it’s inevitable. Regardless of how you feel about AI, it has triggered a paradigm shift in how we approach just about every aspect of product, especially at the leadership level. Which of these new tools do we deploy and how? How will this impact the structure and functions of our teams? And as our organizations change, how do we need to change with them?

My guest today is the wonderful Greg Petroff, a professor, design thought leader, and seasoned executive whose resume includes companies like SAP, GE Digital, Google, ServiceNow, Compass, and Cisco. And as Greg and I were discussing these changing times before recording the show, he made the point that we’ve been here before, with the Dot-Com era, but the difference is the pace of change and in turn, the required pace of adaptation. You’re about to hear Greg talk through the tools, organizational changes, and strategies that executives need to be thinking about right now—because, Bob Dylan says, and warns us very well, ‘you better start swimmin’ or you’ll sink like a stone. Let’s jump in.

Welcome back to The CPO Club podcast. I’m here today with Greg Petroff.

Thank you so much for joining us today, Greg. How are you doing?

Greg Petroff: I’m well, thanks Hannah. Thanks for having me.

Hannah Clark: Yeah. Cheers.

So we’ll start off the way we always start off. I’d love if you could tell us a little bit about your background and how you got to where you are today.

Greg Petroff: Yeah, I am an architect originally in my career. I started in architecture a long time ago. And then I’ve had roles in sort of enterprise software space from product management to design leadership over the last 25 years. Places like SAP, GE, Google, ServiceNow. Compass most recently I was at Cisco and now at this point, I’m doing some independent consulting, a little bit of fractional design leadership and coaching, and I’m teaching at California college for the arts.

Hannah Clark: Busy schedule. So today we’re going to be chatting about some big topics, mindset shifts that leaders need to adopt in order to adapt to this new AI era that we’re all in and trying to figure out.

So to kick us off, in the past we’ve drawn some interesting parallels between our current AI moment that we’re in and the Dot-Com era, which was also a big shift in mindset. So, what are some of the key lessons from that earlier period of tech disruption that are currently relevant in this shift?

Greg Petroff: Yeah, I think there’s a couple of things that are going on. I think one expertise isn’t necessarily as valuable as it used to be. I think that we have a kind of a prediction framework of how we see the world, and it’s based on our experience.

And then when we start to look at work, we use that framework to help us navigate and do the next set of things. And that’s useful when the rules are consistent, but in moments of change, the rules may change and you may not see them. And so curiosity is very important behavior to cultivate right now.

And I think in the Dot-Com era, there were lots of companies that sat on the sidelines and didn’t really understand what it was. And then there were a bunch of people that didn’t really understand what it was, but they were curious and just jumped into it. And started making things and I think we’re in a very similar moment where it’s moving so fast that the way that you have to find your way into understanding how you might use AI is to be using it as much as possible.

Hannah Clark: Yeah, with everything shifting, that also makes me think about how traditional roles traditional as we see them now, obviously traditional to us now is not really that traditional. It’s all changed so rapidly over the past several decades.

But can you elaborate on how you’ve seen some of the changes in role boundaries and product development play out so far?

Greg Petroff: Obviously, it’s situational. So every company has its own culture and strengths, like Google’s very strong engineering culture and, the other disciplines are important, but the engineering kind of drives the car. Other organizations designs more focus in the center and others are product management.

It’s more focused. So, culturally organizations have a bad. Also, I think some organizations are very orthodox about the role definition and they say, this is what you do and you don’t do things outside of that and. This is what your cross functional peer does. And I think that worked for a while.

I’ve never been really orthodox. I’ve been one of those people in blended boundaries in my career. And sometimes I’ve been accused of being a designer who does product management or a product manager who does design. And my perspective is it’s about product culture. And I think one of the things that’s happening right now is that the tooling that we’re using is so different than it used to be.

It’s so much faster and it’s more accessible for a wider audience. And so those of us who might have been in a silo can start to do some of the work that might have been a peer’s contribution. And the question is, where do those boundaries map? And, so the Venn diagram is collapsing a little bit on itself.

And I think, the future is going to be teams where, there’s this term T shaped people, right? So you have, you’re very deep in one area and you can navigate horizontally and I’m almost feeling like it’s going to be a double T, we’re going to get to a point where you’re going to find these sort of hybrids, someone who can design and write some code or someone who’s a product leader who came from design or an engineer who loves product management.

And so I think the more important thing is that whatever organization you find yourself in, that there’s an open conversation around the tools and what they enable and who’s on first, basically, who’s going to tackle certain aspects of the product development life cycle. And it gives us an opportunity to shift from like an engineering led culture or product led culture, or a design led culture to a product outcome culture and where everyone is contributing, based on the skills that they can bring to the table.

And so I think we’ll see hybrid organizations that may look very different along the way, in the project management space, I think, the definition of what to build is going to become much more important because it’s becoming easier and easier to build things. And the incrementalist mindset of just adding a feature or adding a piece of something you can do that, but for the same amount of effort, you could do something that’s much more valuable for end users and customers, and you can imagine things that might have been not possible before and so.

That’s the thing I’m interested in is like, how do we create the space for that cross functional partnership to be bolder in the things that they’re trying to do, because it’s actually more straightforward to actually accomplish that stuff.

Hannah Clark: Oh, okay. You just opened three forks in the road here. We have tooling, we’ve got the changing of roles and how those are evolving and just organizational leadership. So there’s three paths we could go down. I’m going to choose tooling first, and then we’ll double back on the other one.

Let’s talk a little bit about AI tools and product development, because you hinted at the tools are changing, tooling is changing, and this kind of goes down to discernment, because we have a lot of tools available to us, and we’re really at an interesting juncture where we really have to exercise good judgment about which tools do we decide to use and how.

Hannah Clark: So how do you help teams develop that good judgment or that, that kind of gut feeling about when and how to use AI tools?

Greg Petroff: And also, I think there’s an important corollary on discernment. When you are using an AI tool, do you believe the answer that it’s giving you? Or do you have the ability to co-creator, work together with the AI in a way where it’s collectively moving towards the outcome is trying to get to. I think part of it is the experimentation. I mean, the space is moving really fast, but I think there are some immediate areas that make a lot of sense. I think in product discovery.

There’s a whole bunch of tools now that you can do, you can build custom rag models on your research insights and, for organizations that have multiple years of research, you can take all the transcripts from your research and build models that you can ask questions of. It’s almost like having a virtual persona.

I know that’s controversial. And I wouldn’t call it a virtual persona. I would just call it making the knowledge base that you already have more accessible and easier for people to traverse. And specifically when you onboard new members of the team, helping them upscale their domain knowledge because they can have this corpus of information that’s really useful to them.

I mean, it’s not easy to do, but it’s one that once done, and I’ve been experimenting with that with some friends. And, the fact that it comes up with Insights and elements of knowledge that may be commonly understood, but maybe not as clearly described is really interesting, certainly in the design side of the tooling side, you’re looking at, there’s a bunch of things coming in the Figma space, Figma itself is a new tool, it wasn’t here really 5 years ago, and it’s changed the way that we work.

I think teams are working straight high fidelity, and I’m not 100 percent sure that’s the best idea because sometimes the low fidelity allows you to see things that you might not see once you’re looking at something that feels like it’s a completed product, but it’s so much faster. And so teams can iterate and make very quickly so that, when the design team can make faster than you can think, then it’s often useful for product teams to make to think versus thinking to make.

And what I mean by that is the conversation you have with mistakes that you make in the artifact that you’re creating, inform you about the outcome and the customer problem in ways that just talking about it sometimes are insufficient. And, pictures like, a thousand words. What we’re finding is that design teams that work with their product teams very early and just iterate, and they’re not really making the final product, they’re just iterating and making stuff, can basically replace the PRD with a prototype and vet the prototype and actually have something that they know that would be of value in a way that was expensive to do in the past and took too much time.

And so, so that’s an example. And then certainly on the engineering side, I mean, People are experimenting with things like Copilot and tools that allow them to move faster. I think we’re still early days there because I think engineering folks are on the fence about its utility. But the fact that large parts of code bases could be partner written with you.

Also, it frees up the engineering teams to work on maybe harder problems. So all these things are there. And I think one of the things I think we have to have a mindset of collectively is how do we disrupt our work, because most of us are trying to place these tools into our products and we would understand better the outcomes that they could provide by also using them and, our day to day product development processes.

Hannah Clark: I’m right there with you. I think that there’s, right now we’re looking at developing this muscle of using AI tools as an extension of our expertise rather than a replacement for the institutional knowledge that is still critical to have, because we spoke about this before. We’re in the position to oversee AI tools as if they’re carrying out some of the tasks that we would have in our purview and that we’re still accountable to.

So that kind of plays well into our conversation about the value of institutional knowledge, particularly in research teams, I think that research is an area where the use of AI is a little controversial. So, how do you see that relationship between human expertise and AI powered knowledge management evolving?

Greg Petroff: Yeah, I mean, I alluded to it earlier. Most research teams have a corpus of content, and they’ve been using Years ago, it was really hard to share that content. And you do a research project, and you present your findings, and then no one would ever benefit from that content again, because it would go into an archive.

And, also research is temporal, it may have efficacy for a period of time and then behavior patterns and your customer base shift or have new technologies arrive, et cetera. However, in large organizations, there’s usually a lot of content available and not many people actually access it.

And the research team would like more people to access it because they feel like it will drive better insights and those insights will drive better outcomes. So I think one part that’s interesting is building custom knowledge graphs of your research corpus. And then enabling anyone in the organization to ask questions of it.

And that’s not replacing research. All that’s doing is allowing you to see the patterns of insights that collectively you have. And it shares that knowledge over a wider audience in the organization. So it’s not just owned by the researcher anymore. The researcher has done the work and it democratizes access to that knowledge.

In a way, it helps everyone understand who are we serving and why? And then it allows the research team to keep feeding that. Their job is then, at some level, to continue doing the work that they do. And I’m a big believer in qualitative research and even quant methods where, you’re instrumenting et cetera, and not giving that up to an AI to do for you.

The ability to synthesize that knowledge and share it more broadly, I think, is going to be something that’s going to be really interesting. And I’ve always looked at democratizing research. I know people feel controversy around that idea. Whenever I’ve seen access to something that’s a value in an organization become more open and more available, the demand for expertise actually goes up.

And what I mean by that is that if more people see insights, the more they want them. And the more they recognize that they need professional insight gatherers on their team to do the work that, they know how to do well and do it in a methodologically solid way. And so I think that’s the area that I’m interested in is how do we make sure that everybody is deeply seeped in the knowledge of who they’re serving and taking that information to make better products, I think is things that actually make better products.

Hannah Clark: So we’ve covered this topic a little bit as well, because, yeah, this, I think this is a really interesting area of innovation for AI. We had a really popular episode on product discovery that was enabled by Gen AI with Craig Watson, who’s the founder of Arro, some time ago, that was, that really dove into how these tools are really, I like what you said about using a persona, because, like you said, we’re not really trying to just, Replace the persona development that we need.

It’s more about making that data interactive like what we have developed and what we have learned. And similarly, we did a couple of episodes on the product analytics tools that are Gen AI enabled that are really fascinating. We did one with Mo Hallaba, who’s the CEO of Datawisp and also one with Mario Ciabarra at Quantum Metric.

The way they put it is the democratizing of data and making it more available and more interactive to people who don’t have that data scientist competency or that skill set. And I agree, I think that once, once data is more accessible and we can really see, no matter what kind of stakeholder we are in the organization.

We can really understand how those insights benefit us and interact with them in a way that makes sense for our role. Then it really does open up the door for, okay what does a data informed culture, what can it really look like if we’re really using those tools effectively? So I think it’s a very exciting moment in product.

Greg Petroff: Yeah, we’ll see what happens. I mean, I think domain expertise is tightly held. It’s not something that’s universal in organizations. It’s also something that is hierarchical. So in some organizations, the, those who have the most domain expertise have the most influence, I’m not going to say power, but most influence in the organization, what happens in an organization where everyone has access to a deeper understanding of who you’re serving and why I think that part will be interesting to see how that unfolds and if it helps or if it breaks some conventions around, the structures of how we see ourself, that part’s interesting.

The last thing is, I get in trouble sometimes for saying this, but I think the role of research is to de-risk product decision making. And what I mean by that is, yes, we are trying to understand and empathize with our end users and understand deeply the problems we’re trying to solve and what outcomes are important to them.

Product leadership makes assumptions about what features and functionality will benefit end users and also make a great product that people are willing to pay money for. And oftentimes some of those assumptions are loosely held, but are expensive to implement and they’re not vetted in a way where you have some degree of certainty that if you do that, people will actually value it.

And so I think this is 1 of the things that researchers can do is if they want to move up the stack in terms of importance. It’s to be able to obviously give the general understanding of who you’re serving, but also find the parts of the product development process that there are lots of assumptions about that haven’t really deeply been validated and use their strength to provide more signal so that when leadership makes decisions on what they want to execute on, they have more certainty that if they do that work, the expected outcome will be the expected outcome.

Hannah Clark: Yeah, I’m actually surprised that you get in trouble for saying that, but I won’t tell on you. As a show, we are really on the side of the UX researchers.

And we think that they are, they present an immense amount of value to me. I think it’s quite surprising that we, if we’re about to invest a great deal of money into producing a product that we really want to be absolutely sure that it’s going to be, resonant with our target audience kind of alludes me why we would not.

Greg Petroff: The other thing is interesting too. Like we now have access to do full circle, right. Which we hadn’t before. So, we have these tools like Amplitude that allow us to get if they’re instrumented well, get really great telemetry on behavior. We have product success metrics. We’ve got a feature utilization for insight.

We’ve got, in SaaS businesses, we have renewal rates and then we can do cohort analysis over which customers are renewing with the least amount of effort on the sales organization, et cetera, and you can take all those signals and historically, they’ve been information that has been owned by customer success or owned by the team and owned by this corporate strategy team and owned by the research team.

And I think there’s an opportunity for research to see all of that in a big 360. And so that you can get a health check of the experience that you’re creating, the business that you have, the behavior of your customers, et cetera. That was just almost impossible to do before. And it’s not easy to do.

I mean, there’s only a few people I know in the research community have done that. I think that’s where AI can also help is to start melding different data sets on customer success and user success together to get a better picture of the health of your product and then also what needs to happen next.

Hannah Clark: Yeah, I think that’s one of the biggest benefits that we have of some of these conversational models is being able to train them with, all of these kind of siloed outputs and be able to unify them into some, something that was a little bit more usable that we can really ask more pointed questions of, I mean, like you said, we’ll see how it shakes out. I’m sure that there’s, that there’ll be a lot of kinks.

Greg Petroff: I’m going to say one last piece on that too, which is then once you get the answer, another reason why you need a research team is you should have discernment and you should go double check because it may just be, your data isn’t that great. And you’re getting an insight that is really not real.

Hannah Clark: Yeah. I kind of wonder yeah, as we start to rely more on these models, how data hygiene is going to become more part of the conversation about, is the data that we’re feeding into it of high quality?

Greg Petroff: I can’t imagine the moment where you say, our AI thinks that you behave this way. Is this true or not? No, I’m kidding.

Hannah Clark: You touched on some kind of organizational and adaptation themes here. So I want to dig a little bit into that. So we’re talking a little bit about the uncertainty of role evolution, people’s roles changing, things melding and shifting. And then there’s also this core expertise that kind of when we have many roles as an individual contributor or a leader, I think that there’s some questions about what is the core expertise that you’re bringing into the table and how do you defend that when you’re in a situation where other people have overlap with what you’re doing, what you’re contributing?

There’s also AI tools that contribute some of that. So how do we help teams navigate that uncertainty of role evolution and maintaining their core expertise and value and kind of protecting like the area of expertise that they’re bringing to the team?

Greg Petroff: Yeah, I almost questioned the last statement protecting your turf.

I’m not sure that’s what you meant by it, but I have probably changed my career like, 11 times in the last 25 years as things have evolved. I started as an architect and it was 3 graphics and then I was in. Broadcast media and then it was in Dot-Com and I was information architect and I was an interaction designer.

Then I was a product leader and I was a researcher. Then I was back to interaction design. Then I was a UX leader. You and I think 1 of the things interesting about this moment is we’ve actually had stability in the roles for the last 10 years. And so a lot of people have grown up in that era.

And have a solid definition of their value, and a solid definition of who they are, and a solid definition of what they do. Haven’t experienced having the rag pulled out from underneath them because there’s some new way of doing things or a new piece of technology. Some of us have been around long enough, it’s happened to us so many times that we just see it and say, oh.

Okay, the rules just changed, okay, now we got to be more adaptable. And I say, expertise is really important, right? You know what I mean? So I think you should know what you’re great at. And I think the way I sort of coach people is lean into your strengths and value them. But don’t get caught up in your role definition, we’re moving into a moment where it’s more about the outcome and the problem definition.

And then the execution, there are a lot of different ways to get there. And if we hold on to the orthodoxy of our role or position, we might just be fighting the currents that are around us. And so, stay curious, have a mindset of evolving and constantly, learning, and at the same time, don’t shift away from what you love and what you’re good at, because the best teams are made up of people who are exceptionally good at something.

I’ll give you an example. When I was at SAP, this was almost like 15 years ago. I was part of a team. We weren’t a design team. We weren’t a product team. We weren’t an engineering team. We were an innovation team. And it was made up of, we had anthropologists on the team, we had engineers, we had product managers, we had designers, and we would be tasked with a project, and we would assemble a team, and the team was always an eclectic group of people, and we would just look at the problem and figure out how we were going to work on it together, and I anticipate there’s going to be more of that kind of behavior in the future.

I think in large organizations, you have to organize in a certain way, because It helps your product development process, but in smaller organizations, I think that we’re going to be looking at an environment where, we’re just gonna be more nimble about the tools and the things that we have.

And if we stay orthodox about what we do, it’ll actually get in the way of actually doing the work.

Hannah Clark: Okay. So I’m glad that you brought up a departure from orthodoxy, I want to talk about leadership styles. So you’ve observed that there’s a bit of a generational shift right now that’s happening in product leadership styles.

So we’re moving away from this very command and control top down approach to something that’s a lot more collaborative. Now that we’re on this anecdote train, would you mind sharing a specific example of how a shift like this has impacted product development outcomes that you’ve observed?

Greg Petroff: I don’t know which strain of product development will be a last or if they will both be there in the future, but there are certainly two models. There’s one model of the really strong product leader who is not 100 percent transparent with the organization and it’s making moves along the way to move the project forward.

And it’s a very much kind of a command and control kind of environment. And there are many organizations that operate that way. And historically, there’s been some incredibly successful people in the product management space who behave that way. So many earlier career product leaders will look at those people and say.

I want to be like them and I’m going to do the same thing the challenge I’ve seen, especially in large organizations, is that there’s a behavior that can happen where if resources are constrained and the incentives are set up for you to deliver product leaders will compete with each other for resources.

Because if they over deliver, they get the next job promotion. If there’s no one at the top sort of saying, these are the things that we’re not going to do incentivizes people to try to get more done in the time that they have. And what they don’t recognize in that moment is they place strain on the system because there’s only a finite set of resources.

And so they’re actually using those resources less efficiently because they’ve put too much into the system. And therefore, you end up get a bit of crazy town, and then people move into kind of command and control to try to whack a bullet into place. I’ve also seen the other side of it, where you’re seeing people who are like, hey, I’m entering this project, and I don’t know what we’re going to do.

I don’t have to be the person with all the answers. But I’m going to assemble a team, and collectively, we’re going to learn together as quickly as possible, and we’re going to build a set of experiments along the way that allow us to learn as quickly as possible, and we’re going to collaborate and share that information.

And the project leaders who do that, I find, get more out of their teams, they have more clarity, there’s more visibility on the work, people feel like they have autonomy in that, they can find their role in the work and the effort. And it may be an indirect observation, but it feels to me that generationally, it’s more earlier career or younger PMs who behave this way.

And I think it’s partially because they’re digital natives to begin with. Some of the more senior people, aren’t really if you look at demographically and have grown up with great software and want to create a space for value for the design team and for the research team and for the engineering team and the product team to collectively solve a problem together.

And, I was at the Canadian friends conference this fall, and it was just really fascinating for me to hear the conversations around how that community of product leaders thought about how they worked with a cross functional peers. And it was really heartwarming for me because it was a very open conversation.

It was that it was a 1 where people were willing to be vulnerable about that. They didn’t have the answers, but they were organizing their team to go find the answers and. Yeah, that’s a little bit different from feeling like you have to have the answers and your team’s counting on you to have them. And so you fake it till you make it and pivot a lot until you get there.

Right. And so I think there’s sort of two strands that are happening that product development community that I would be curious to see if, better outcomes happened, one law or the other. I don’t have great examples for you, but I just, these are observations of behavior.

Hannah Clark: I tend to agree with you though, because I’ve observed that work culture in general has become a lot less hierarchical and a lot more collaborative. And I really like that shift. I think that it’s, we’re going in a really positive direction in which vulnerability, attention to psychological safety with your teammates, these kinds of the concepts of emotional intelligence that are making their way into the leadership sphere.

And are really starting to have this nominal effect on how people lead teams and work with people cross functionally is it’s really heartening to see how that’s really enabling a lot of people to do some of their best work.

But I’m gonna put you on the spot a little bit since we’re talking about this, you’re a leader, you’re a seasoned executive, and you’ve managed a great deal of teams. So what strategies have you found to be very effective for fostering, as you mentioned before, curiosity, experimentation, while maintaining this high standards of quality?

Greg Petroff: Well, clarity is the biggest gift you can give a team. You can’t be transparent about everything in leadership, and it doesn’t actually help people because, there are a lot of things that happen in the life of the company that sometimes are transactional or are a little bit hard to understand and communicate. And then if it, if everybody’s talking about it, then no one’s focused on work, but clarity is really important, helping people have ownership of the, of that clarity, right?

So it cascades down so people can see their role and how it adds up to it. It’s hard for organizations to say no to things, but I think the organizations that declare what they’re not going to do are, very, it’s useful because it stops eager beaver behavior around Hey, we could do this, or I have this pet project that I want to do.

And if you say, no, we’re not doing that doesn’t mean it’s not, we’re not ever going to do it just means we’re not going to do it now. So that people, again, that’s a part of that sort of kind of clarity aspect. I think giving feedback in a courteous way, but also holding people accountable. I think the notion of creating environments where people feel safe makes sense.

And it’s incredibly important. But you also have, you can’t shirk away from giving constructive feedback. And when I was at Cisco that one of the tenants that we had that I really loved, and it came from Duo, which was a startup that Cisco had purchased about five years ago, and they had this tenant that they call be kinder than necessary.

And what that meant was if you had spinach on your front teeth and you’re walking into a meeting, you’d want someone to tell you that you have spinach. It’s not about you. It’s just that this is something that you should know about. And so I think it’s about constantly giving feedback to each other and holding each other accountable to a high standard and making it about the work, not about you.

And that creates the conditions where people learn to listen to that feedback and don’t take it personally. They take it like, Oh, okay, that’s an opportunity for me to improve my game, or that’s an opportunity for the project work to, to be in a different direction. So I think those are the two things that for me.

I’m not saying I’m great at it, but I materially try to practice is try to be as clear as possible and try to give as much feedback as possible.

Hannah Clark: I really appreciate those insights. I agree with him. I’m sure you’re just being humble.

Thanks for joining us today, Greg. This has been a great conversation. Yeah, I’ve talked about a lot of different topics that are really highly relevant right now. So I’m hoping that we get some interesting feedback from folks who are listening. And speaking of feedback, where can people follow your work or connect with you online?

Greg Petroff: Yeah, I’m on LinkedIn. So you can find me there. I have a profile there. I have a Substack called Improbable Futures. So I do a little bit of writing on the idea around possibility. And so you can find me in those two places.

Hannah Clark: Awesome. Well, thank you so much for joining us here.

Greg Petroff: All right, thank you.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides, and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The CPO Club, wherever you get your podcasts.