How To Stand Out As An AI PM

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: I’m just going to be blunt—whether or not you’re making it an explicit career goal to specialize in AI product management, it is now a skill set every software PM should be developing. I think it’s safe to say that now that the technology is here, it’s only going to get better, more intelligent, and more ubiquitous—so learning how to work with AI is about as essential as learning how to work with the internet. In fact, you could argue that basic skills in AI are already a minimum requirement when applying for PM roles. So how can you take it a step further and stand out in the AI PM job market?

My guest today is Praveen Gujar, Director of Product at LinkedIn. As the title would suggest, Praveen has an excellent vantage point on the growing use cases for AI in the digital product industry, as well as an intimate understanding of the breadth of skills product managers need to develop to truly succeed in this space. We discussed both the technical and theoretical aspects of leveraging AI, but if you take away nothing else, our conversation on data hygiene—which comes up about halfway through the episode—is critical info for anyone hoping to not just land an AI PM role, but excel in it. Let’s jump in.

Welcome back listeners to The CPO Club podcast. I’m Hannah Clark, and we are a community of tenacious product leaders here to share the weight of scaling your product. Our members are a SaaS-focused PMs who genuinely love their work and are driven to help others succeed. So if you want to hear more about that, you can head over to theproductmanager.com/membership. We’d love to have you on board.

And today we’re speaking with Praveen Gujar. Thank you so much for joining us.

Praveen Gujar: Glad to be here. Thanks for having me, Hannah.

Hannah Clark: Yeah, pleased to have you.

Can you tell me a little bit about your background and how you arrived at where you are at LinkedIn today?

Praveen Gujar: My background is mixture of engineering and product management. I started my career out of the engineering program as a software developer. But I soon realized that I think working backwards from customer and building great products is my niche thing. So I got my MBA and started with Amazon as a product manager.

TLDR, my experience, it’s in two key areas. One is in digital advertising and second one is public cloud. And all these products that I’ve built over these two segments are AI powered. I have a tenure in Amazon, LinkedIn, as well as Twitter, so built these products in these large organizations. Yeah, been on the AI PM for a long time now and happy to share my experience and my expertise in the space.

Hannah Clark: That’s so fantastic. And I know that we’re very keen to hear a little bit more about that. So today’s topic is going to be specializing in AI product management. And this is obviously a huge field that’s taking off enormously right now.

So to kick us off, what are some of the skills that separate a great AI PM from a traditional PM?

Praveen Gujar: That’s a great question. First and foremost, if you want to be a great AI PM, you need to be a good PM to begin with, or a great PM to begin with, right? A lot of the skill sets that are necessary to be a great PM are a must-have to be a great AI PM as well. That includes like really identifying the problem, working back first from customers, really having a product sense to build the right product and driving the product executions phase as well as the go-to market phase as well.

All these things are key necessity. In addition to that, I think being an AI PM, there are three key things I would say are very critical. One is technical understanding of the AI space. I think it’s very essential to be a good PM there. I think you need to understand how the machine learning and AI concepts, the models, the different types of models, including basically training of the models, as well as how you basically interact with your AI engineers, like, it’s very critical in that respect.

The second piece basically becomes the data science or the data angle to it as well. As a PM here in this space, you need to understand data really well. Understand how the data flow happens, how the data is used to train the models, and what does clean and hygiene data means, and how data can introduce biases. All these understandings are critical for you to be a great PM.

Third, but not the least here, I think programming can be really a value addition for a PM as well. So if you know how to program in Python and R and other areas, you can relate much better to the engineers who are actually working on the space, then you can be a lot more effective as a PM.

So I would classify these things as more of a technical expertise in AI space. So that covers one aspect of it, right? The second aspect is applying these AI concepts to your domain. So if you’re working in healthcare, if you’re working in financial services or any other domain, how do you apply the AI principles that you use, the AI technology that you are so familiar with in making the products more resonant with your customers?

So that’s a very key aspect of it of being an AI PM. Third pillar, I think I would say like, you would need to up level your PM skills. One of the key aspect becomes is inexplainability. How do you explain this to folks around you, including execs who may or may not be AI ready or AI proficient. As well, how do you do that is very key aspect.

And understanding the entire model lifecycle management becomes very key as well because you are working with engineers and fine tuning the models and even going through the entire lifecycle of the models and understanding it and basically working a lot more closer with them becomes very key as well.

So, to summarize the technical expertise, then applying the technology to your domain area becomes and then like uploading your PM skills for more explainability and other aspects becomes very key. This is what makes a great AI PM.

Hannah Clark: Yeah, and I’m sure that everyone is going to be really clamoring to develop those skills because this is just such a huge area of opportunity.

And speaking of which, from your vantage point, what’s the outlook right now for PMs with skills or experience in AI?

Praveen Gujar: AI has now become a household name. I think most of the attribution there goes to ChatGPT and its popularization, but AI is not a new thing, right? People in technology industry have been using AI in various ways.

I do come from a digital advertising background and AI has been powering various systems and capabilities for a decade plus right now and other in other fields even longer, I would say. Like, if you look at what in the last two years has been is an exponential raise in both popularity and usage of AI, that basically opens up a huge amount of opportunities for PM with AI skill sets to work on different domains, different problems, and different creative or innovative solutions as well. So that explosion is a very key thing and it’s going to continue in the near future. That’s the outlook, at least from what we see.

Second is the expansion of the role as well. So a lot more and more in the last two years, every product and service organization is going through some kind of evolution of adding AI touch to their product as well. This may be enhancing their experiences with AI or rebuilding their products with AI first or AI integrated approach.

So that opens up a huge opportunity for all the PMs working in the space to really work on some cutting edge technology and add real value to the customers. And also, like, it’s strategically important these days, right? Like, I think you may even argue that it is from a PR perspective, important as well.

And so is important in your profile building if you are a PM. But more and more, this is more strategically important from an organization, so you are better positioned in the organization as well as a PM if you are a AI PM, because you are valid more in terms of because you have skill sets and you are strategically more aligned to the company needs.

Last but not the least, I think there is a need or a demand for AI PMs because of the explosion that we have actually seen. So you get to be a bit more in demand for your skill sets. That opens up all new opportunities from a skill set development perspective, from building awesome products and also even monetary benefits as well.

Hannah Clark: A lot of opportunities, a lot of benefits.

So let’s go back to talking about the skills and putting those into practice. What are some tips that PMs can use to leverage AI to build high growth products?

Praveen Gujar: I think that’s a great question. So we are even publishing a paper on this in engineering management review, really thinking through what does it mean to build a great product in AI, right?

If you look at the current trend, what is happening is everyone is jumping onto the bandwagon. So you may see a lot of organizations enhancing their products and services with AI touch to it as well. Many of them are mostly superficially done. So for example, it can be running a chat bot on existing services.

This is like very similar to what are mobile era taught us as well, right, when mobile became so ubiquitous, then the first reaction organization had was shrink your existing product into the mobile format that didn’t really last for long, right? Similarly, I think what we are seeing is a lot of organizations, for lack of better imagination, they are implementing or sprinkling AI over their existing product and solutions.

That’s not a sustainable mode for organizations to carry forward. What we see is where there is a sustainable advantage is when you actually build AI first or AI integrated organization products. What do we mean by that, right? Like, it’s basically making AI the center stage of the product and building the experiences that are powered by AI, that are fully automated by AI as well.

Instead of basically just leaving AI to automate a particular workflow, empowering the AI models to make optimizations is very critical. The second part basically is decision making as well, right? A lot of AI, what is basically used because of the recent trends is, how do we use AI to answer a quick question?

But the core optimization and decision making is not something that, that is something which will truly enable the product to be a lot more proficient. Last but not the least, I think the organization structure becomes very important as well. If you have a more embedded team with AI engineers working very closely with product managers, what you’re producing are more AI integrated products thought through from scratch that can actually enable you to compete better with your computation and also build more sustainable mode in the future. So I would say like these things are very important. Like, how do you think about reimagining your product and services with the AI centered approach?

And then solving the user problem, like the user problem solution, like solving that is a paramount need for any product, right? There is no shortcut there, but really reimagining your product with the AI making the decisions basically applying a technology to improve the current experience is what is needed.

Hannah Clark: I’m curious about some of the use cases that we’ve seen powered by AI currently in the space. So obviously there’s some, I think that we could all agree that there are a lot of products that have been enhanced significantly by AI and a lot that maybe not so much. What are some of the best use cases that you’ve currently seen or that you think that there’s a huge opportunity to expand on?

Praveen Gujar: Yeah, so I come from the digital advertising space. Let’s take an example there, and I’m happy to dive deep as well. What are the key things in digital advertising, right? You need to ensure that you have a right audience that you’re targeting to. You have the right content for those right audience, and you serve them when, where they are at this point.

So I think traditionally how this approach used to happen is we started with the AI models really trying to understand what is the best audience fit for your product and services. And depending upon your core objective, how do you target these specific audiences? Over the course of time, we have improved those and a lot of predictive algorithms basically enables us to understand what are the predictive audiences that you can go after as well.

So that’s one use case. With the advent of Gen AI, especially in the last two years, content creation has become very key. For example, I think if you are a brand, if you want to basically reach to different set of audiences, how can you generate 80% of the content that you basically have in 80% of the variations in a way faster and quicker that basically enables your creative team to move faster. And then basically be more productive at the space has been very key factor from the AI space as well.

And then like you have now the audience, you have the content to go after, how do you personalize this at scale, right? Anyone can personalize when it’s a small set of population, but if you are a large organization trying to reach millions of users with personalized content, AI enables dynamic audience segmentation as well as content creation so that you can actually personalize your message at scale as well.

So those are three different pieces that were being developed for a long time as well. Now, how about stitching all them together as well to make sure that it’s completely automated and optimized across these different areas as well. And this is where you are seeing a lot of the advertising platforms are providing one click fully automated end to end automated solutions for their customers.

Be it Google, LinkedIn or Meta, you see a lot of these automated platforms emerging up as well. So that’s just one digital use case, right? I think we keep talking about the use of AI in healthcare. One of the things that I know a friend who works in Google who specializes in scanning through X-ray images to identify anomalies that can actually cause cancerous aliens as well.

The models basically that has been developed over the course of years have been increasingly proving to be a lot more effective in identifying early trends in malignant tumor or anything much earlier than what a radiologist or anyone can identify as well. So these are great examples of how AI has improving not just productivity, but the effectiveness of the work that we actually do.

So I think these are probably just a couple of examples of what, there may be thousands of such use cases.

Hannah Clark: Yeah, I can only imagine every industry has probably got something to that effect.

Let’s talk a little bit about data hygiene. So what’s the importance of keeping clean data when working with AI tools?

Praveen Gujar: I think you hear quite a bit about how these large language models or any AI models are trained with large and large amount of data as well, and it’s kind of a garbage in garbage out kind of a mechanism as well. If your data basically is not of higher quality, and it’s curated for biases, incorrect accuracy, then I think the output that you basically get out of these models will be suboptimal as well.

So that’s why data hygiene plays a very critical role. I think there are, in the earlier point that I actually made as an AI PM, you need to understand the data, and you need to understand the data platform part of it as well, right? I think that’s very critical piece here as well. One, basically improving the accuracy of the model that you’re actually working with is very important.

Minimizing biases is very key in the AI model and that’s data hygiene becomes very important as well. And the performance of the model, that’s basically accuracy as well. But also, the training of the model is a laborious work still, and you can spend hours and months together in training of the model as well, and any bad quality data make the process lot more tedious as well.

Last but not the least, I think we can expect increasing regulatory scrutiny on some of these AI models and the output they actually generate as well. And if the source of these tends to be an unhygienic or uncurated data, then the output of the model basically may not meet the regulatory standards that these bodies may actually enforce on in the future as well.

Take a very simple example, if you are in a financial sector industry and you are advertising for your customers. And if these advertising models are not trained with high quality data, they may inadvertently introduce biases. And that may also mean that you can get more regulatory scrutiny because these biases may be towards an underprivileged community, for example, that violates your fair lending act.

So this is like a cyclical effect that you actually go through. So that’s why the impact of not having hygienic data basically is a lot more than what you can realize at first. So training the data, like first of all, having systems and platforms in place to curate the data, get the data in a way that models can be trained more effectively is very important.

Hannah Clark: So this actually is an area that overlaps with the ethics involved in using AI.

So I’m interested to hear your take on ethics in general in the AI space and how can PMs who are new to the space ensure that they’re acting responsibly when implementing AI solutions.

Praveen Gujar: I think it’s a great question. I think something that is paramount importance for so many different organizations, in fact, like a lot of these organizations are building up teams that are called responsible AI teams, right?

It’s particularly based and geared towards making sure that the ethical and moral standards are actually maintained. I took an example of this financial sector. This is a great example of how, when not followed right, can have a societal and economical impact specifically, maybe on certain sector of the population.

So if you’re an AI PM, it really starts with, first of all, education, educating yourself about some of the shortcomings that you can actually have. Unhygienic data, for example, or if your models are not trained the right way, like educating yourself from all these things begins the first thing to begin with. Then, basically, the second part you have to focus on is when you find out that a particular data has not curated or can introduce biases, how do you basically, what are the steps that you actually take to clean it up to make sure that you retrain the models and validate the accuracy of the models after you retrain the data that actually becomes important as well.

And the third piece of the also is, I went back to the principle of having building the right organization structure. So having a diverse team also becomes very important as well, even as I would even recommend having a dedicated POC that the teams who can actually play the role of diversity are upholding the ethical standards in the team can be useful concept to apply on.

So you can actually have the person monitor, be the champion for making sure that we clean the data. We train the data in an ethical manner so that the accuracy of the models are not more relevant. So all these are the best practices you can actually apply as a PM in the space to ensure that the the ethical standards are maintained.

Hannah Clark: So switching gears a little bit, what are some of the ways that AI can help us run more efficient experiments?

Praveen Gujar: Yeah, that’s a great question. So think about the traditional way of running experiments, right? So if you are a say, DoorDash, you basically may be running thousands of experiments in a day.

And you have multiple business metrics and system metrics that you have to monitor as well. If you are a PM, it’s typical that you are running two or three experiments parallelly for different features that you are running. Now you have to comprehend, like, what are the implications of these experiments?

Work very closely with the data scientist that is allocated to you to fine tune the experiment and get the readout and then basically work again to fine tune the experiment, make a decision on whether to RAM, whether to DRAM and also like finally generate the readout and what is actually necessary to to either ramp up the product or even demonstrate the impact of the feature that you are actually developing as well.

So a lot of these things are very human based approach as well. And now imagine you have built an AI system that can actually optimize your experimentation capabilities. That’s going to basically not only make your job as a PM a lot more useful, but also the process of running experiments, evaluating them and making decisions becomes a lot more easier for the PM as well.

It really starts with even say, like when you’re designing an experiment, what are the optimal parameters to consider and what are the hypothesis. AI algorithms can actually help you to come up with hypothesis and select the right parameters to run that experiment as well. So when you are running the experiment and then dynamically adjust the ramp of the experiment or even between variants to find optimal balance in running the experiment as well.

You can even make decisions on whether we have mitigated the risk in this initial ramp and then ramping up for more statistical signals. These are something that they can actually decide more autonomously as well. Last but not the least, identifying anomalous or anomaly within your business and system metrics because of the impact of experimentation is a really a value added feature as well.

So you can identify the impact much sooner and then you didn’t make a decision. The models can make a decision to either stop the experiment or rammed down the experiment before there is a wider impact on the business metric. So, for example, because of you launched a feature and that basically say, broke some sessions on your app that may directly impact your revenue so the models can identify much faster than human beings and did on this experiment to make sure that the issue that was caused by this experiment is mitigated.

So a lot of these ways like AI models can definitely help you experiment better, be more productive and also make timely decisions.

Hannah Clark: Well, before we wrap this has been a ton of information and all very useful. I’m sure that folks are going to be looking for next steps because I know that everybody listening is very excited to start to develop skill sets even slowly into the AI space.

Do you have any recommendations for folks who are interested in developing these skill sets? If they’re not currently actively working on an AI product, any resources or courses that you’d recommend?

Praveen Gujar: Yeah, I think there are plenty of courses on LinkedIn Learning, for example, or in Coursera as well. One of my favorite is DeepLearning.ai, if you go to that website, by Andrew Ng, his courses are available on LinkedIn Learning and Coursera as well. I think building the foundational understanding of AI is very essential. I cannot emphasize the importance of that, right? Because if you have your right foundations and you can then build upon that to understand more advanced technology and then how to apply them in your application.

So what I would recommend your listeners is to basically go understand the foundations first and then build upon that by taking more advanced courses as well. Podcasts can be a really good way to, one, learn and also keep in touch with what is happening in the industry as well. Because this is such a fast and evolving technology right now, so keeping up pace is really difficult and often podcast serves as a really good mechanism as well.

Third, there are multiple blog posts that gets published and blog posts can be a great mechanism to deep dive into a particular topic and understand the practical applications of the topic in your field as well or in general in in any field as well. I think these three can be really good source to go.

There are courses that universities provide, for example, as well. The effectiveness of that, I’m not very sure about it, to be honest, but it’s something that gives you more structural approach. And that gives you kind of a diploma as well to break through into the space that can actually be a very good asset as well to begin with.

So, yeah, I think these are some areas where you can either overhaul like just refresh your skills or get to know AI more better to break into this field. These are all can be very valuable valuable things. So I just add to one thing, more than anything else, I think you need to be curious as a PM in the space so you can actually really invest the time and really understand and dig deep. So make sure that you learn the concepts and then apply that in your field.

Hannah Clark: That’s really great advice. Well, thank you so much for joining us today, Praveen. Where can folks follow your work online?

Praveen Gujar: People can follow me on LinkedIn, and I’m actually trying to basically build my own website as well. Hi, it’s praveengujar.com. It’s still under construction, I would say, but all my publications and upcoming book will be publish on that website as well.

Hannah Clark: Great. Well, thank you so much for coming. We really appreciate it.

Praveen Gujar: Thank you so much for having me.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The CPO Club, wherever you get your podcasts.