Slik blir du AI-produktleder uten erfaring (med Dr. Nancy Li)

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: Fair warning, this is about to be yet another episode on AI. I suspect there’s two camps of listeners right now; folks who are ravenous for intel on everything AI-related, and folks who are like ‘AI? More like A, I’ve heard enough about this stuff.’ And if you’re still listening even after that atrocious pun, I’m going to assume you’re in the first camp—meaning you might be the perfect candidate to specialize in AI product management.

Today I’m joined again by Dr. Nancy Li, who apart from being an absolute force to be reckoned with, is the founder of Product Manager Accelerator, the host of the Product Insider Podcast, featured in Forbes, and also a veteran of AI product management. Dr. Li has had a really unique vantage point on both the AI product space, as well as being very close to the product career landscape for the past several years. So, she’s here to share some insider information on how you can leverage your fascination with AI into a life changing career play. Let’s jump in.

Welcome back to The CPO Club podcast. Dr. Nancy, thank you so much for joining us again today.

Nancy Li: Thank you for having me join your show. I had a great time last time filming those episodes, so glad to come back again and share more AI insights.

Hannah Clark: Yeah, absolutely.

So for those who didn’t catch you last time, can you just give us a brief recap of your background and how you ended up where you are today?

Nancy Li: Yeah, totally. So, hey guys, I am Dr. Nancy Li, a director of product feature in Forbes. I’ve helped thousands of people land their dream PM job offer in fan companies and unicorn startups and continue to get promoted as a product leader. Currently I run my own company, which is PM Accelerator, which helping people to advance a product management career. And I also have my own podcast, which is called Product Insider. We feature product leaders, such as VP of product from Google and tell you everything about inside the secrets of building successful product.

And myself was an immigrant who moved to us with $800 in my pocket and became a director of product was in four years. So my goal is really to bring product management education, especially those entrepreneurial education to people from middle income families. And so I’ve been sharing lots of my free resources and secret on YouTube.

And now my YouTube channel has over 1 million views. And also very lucky last year, we successfully started our own nonprofit called PMA Kids, which would give those free product management education for kids from low income families. So I’m very happy to see people have been growing with me and personal growth.

And also the community have been growing over 100,000 product managers through all my social outreach.

Hannah Clark: That’s such an impressive resume. And to top it off, you’ve also built a very successful and award winning AI product as well, haven’t you?

Nancy Li: Yes.

Hannah Clark: Can you tell me a little bit about that?

Nancy Li: Yeah, exactly. So that was before AI became very popular.

The product I built is called Smart Cities product and using AI, especially machine vision, to reduce car crashes. And specifically that product that was in 2016 at the time, there’s a very famous movement was in different cities is called Vision Zero means they want to reduce traffic deaths to zero.

That’s why I call it Vision Zero. So, at the time, we’re just trying to figure out how can we help cities reduce car crashes in a more ultimate way and also more scalable way. By the time, as I mentioned, nobody really reinforcing that you have to use AI. No, we’re just looking at different technology to figure out, well, maybe AI, maybe IOT, maybe data analytics.

I don’t know. So, and we end up decide to use AI as one of the key technology to create such product and wish eventually received the mayor’s best practice award. It was quickly deployed in all the like top 10 cities in the U.S. I’m very, very happy to see how AI has been changing people’s life years ago before it’s very hyped up right now.

Hannah Clark: And that’s a perfect segue into how AI is changing people’s lives today, which is through, in our case, specialization in AI product management. So just to kick us off, what are some of the concrete benefits that PMs can expect if they lean into AI as a specialization?

Nancy Li: Yeah, great question.

So regarding AI as a specialization, it’s definitely opening up so many new doors for current product managers and also anyone who want to break into product management. And for example, one thing is definitely higher salary compared to AI PM. AI PM in general getting paid about 10% to 20% higher than average PM out there.

And second is, it actually gives you a career boost. And for example, I have a student who joined Amazon as Gen AI PM. She literally was moving her career to the next level. And not only she joined fan company, she also was able to build those very impactful product for Amazon that can be used by lots of people out there.

And the third thing is how much it can really accelerate you breaking into the product management space. For example, I have students, they decide to specialize in AI and after they put lots of AI keywords, working on those different kinds of AI product as a project on the resume, they literally get way more interview opportunities compared with any other candidate.

And just because all companies now they want to put AI on the roadmap. Well, we can discuss later regarding should or they shouldn’t, but it’s a trend out there, so it’s definitely pushing them to the next level. And there’s also something very interesting, which there’s a comparison, which is similar to American way to say that it’s like shooting a gun in a barrel.

So if you joined AI space right now, it feels like shooting a gun in a barrel, because in the coming five years, maybe ten, maybe even two years. Once AI to say ‘take off’, regardless you’re doing okay work, hopefully you’re doing outstanding work. Let’s say you do medium work in the space because the whole space is taking off like a rocket ship.

You are riding the rocket ship. That’s where absolutely your career is going to grow just within the entire space. So I recommend everybody really look into AI as your next career move.

Hannah Clark: Would you mind walking us through the AI PM lifecycle?

Nancy Li: Yeah, great. Let me also use my own personal product as an example.

Specifically, the AI product management life cycle has four steps. Number one is ideation and also understand if AI can really solve the problem or not. And step number two is understand what kind of data collection, where do we find the data, because data is a new field of the new AI era. And number three is also building MEP, and number four is launching the products. So now let me use a real life example, share with you guys how you’re able to build the outstanding, in my case, award winning AI product using this framework.

So step number one is really to understand, do customers really have a pain point or we are using AI to create pain point for customers? So it’s reverse engineer. So that’s why, so it’s more important to do very thorough evaluation of the market space. Understand any kind of problem that can actually solve by AI and some problem actually cannot solve by AI. Let me give you a specific example. A case study I did was in PM Accelerator with my students.

We’re doing brainstorming session. For example, if you’re a CEO of Delta, what would you do in the upcoming 10 years? And of course, my students, including myself saying, Hey, maybe you can do some AI. However, the specific AI solutions my student created in the class was saying, Oh, maybe we can use AI to do flight navigation.

I was like, No, that’s the wrong application. Because AI today, especially gen AI, many different AI, there is a likelihood they have, let’s say 1% or 0.001% of likelihood is going to fail. Once they fail, you’re going to kill people because it’s flying navigation. It’s too much risk for people to take on right away for mission critical control application.

So therefore, we must do very in depth research regarding, do we really need it and how AI is actually to improve the application significantly? When I did my own smart city product, as I mentioned earlier, that was in 2016, there was no pressure to use AI. Nobody even thought about AI, we’re just doing real genuine customer discovery. And then we discover AI is the best methodology, best tool to use it. So therefore, definitely do a lot of discovery first.

So second phase of product management lifecycle of AI product is actually data collections and also training of a data model. Data is a new field for the AI space. That’s why some company like Scale AI and it’s very popular right now and many different company, different startup was raised in this entire AI era.

So now when you train any kind of model, it’s more important for you to find the right type of data. And sometimes you can leverage other people’s existing model. For example, there’s a lot of very good AI models out there, for example, like ChatGPT of course. And there’s also different type AI models.

And Gemini has those ones and Soplic has their own AI models. There’s many, many, many AI models out there. So for you, you need to decide if I leverage the existing data model, what’s my decision making process? Which one is better? And maybe you can do API integration. If so, how much is those like costs to me if I directly run on top of other people’s model? Maybe some of the models you can leverage other people’s and then you build something small on top of those to reduce your total cost.

So there’s a many different kind of decision making process, train a model, collecting data and leverage existing model as well. And the third part was building MVP. So building MVP straightforward for those AI product management and very similar to our today’s product management. When you build MVP you must test out with the market, but something special with AI product management is that we need to think about safety and trust for AI product. Because today, not everybody want to adopt AI and then all those different kind of biases can go into AI and those governance for AI need to put into consideration when you build those products.

And ideally, you need to consider those as early as possible in your product development lifecycle stage. As an example, when we build our Smart Cities product, the way we build our MVP is extremely interesting. So we just work with one city, which is Boston, and the goal is to use machine vision to reduce car crashes. However, when we build our MVP, it’s actually very hard to verify, train our data model use limited car crash data.

Think about this. If you want to train an AI model, you need large amount of data. In this case, we’re trying to reduce car crashes. How many car crash data are there out there? First of all, that can be captured and we also need to clean our data and on top of that, sometimes we also want to test out, oh, can our model detect car crashes in snow environment because Boston has six months of snow, by the way guys from Boston.

So in the six months of snow season in Boston, can you detect car crashes? Literally in those scenario, it’s very hard to collect data. We don’t wanna collect those data either. We hope there’s no such data ever exists. Right? So when we build those, the MVP, and we took very different approach. For example, we did two things, and number one, we directly purchased some data.

At the time it is called Mint AI, where we’re able to purchase different kind of like training data from them. And second, we also create those different kinds of simulated environment. Maybe we can do it in the parking lot. People pretend getting closer to cars. And we also create those toy base. Very, very funny.

We have little toy cars, hitting toy bicycles, everything’s based on toys and the toys hitting toys. And, but you, in your AI model, you, of course, you change the parameters to understand the size of the real car. But you can see, can this toy hitting another toy, can AI detect the collision and even predict the collisions before it’s happening.

There’s many different activities we did when we build those MVP. And I also like to give credit to Jensen Huang from NVIDIA. And actually we were early adopters of NVIDIA’s training model at the time, even during that time, 2016, his team has created amazing AI model to do object detections for us to build on top of his model.

So this all those details goes into building MVP and this was very exciting. It’s more than just people think, yeah, just train some model is that you need to be smart and creative in creating an MVP as well. Final stage, of course, is go to market strategy. So go to market strategy for AI product. It has a traditional GTM process, which you have it already.

I have a different YouTube video regarding the GTM strategies, you guys can check it out to my YouTube channel later on. But on top of the traditional GTM process, we must think about additional things such as the adoption from customers and adoption from internal stakeholders. Adoption from customers coming from the risk tolerance.

Some people just do not want to give a try of AI product and could also be people had bad experience once and never come back again. And also afraid to take the data was able to be taken by AI forever. And also felt like, well, maybe AI take over my work, my jobs, right? So those kind of end user adoptions.

We need to really manage it. Then second part, which is internal stakeholders. And through all my engagement with product managers out there who building successful product, we found out not everybody was in the company want to use AI, could be your executives, could be other engineering team, could also be legal team, so they will have pushbacks for sure.

So you need to learn how to advocate those AI product in different scenarios. So those are the four steps, AI PM lifecycle.

Hannah Clark: That’s very comprehensive and also sounds like such an exciting space to dabble in.

So what are some of the steps that PMs can take to qualify for all these exciting AI PM roles?

Nancy Li: Yes, it’s definitely very, very exciting. And I talk about this about the only four ways to become an AI product manager. Number one, which is somebody who are fresh out of school, actually the easiest. If you’re fresh out of school, do anything about AI and just get in first. You have no experience immediate for your capstone project from school.

You can immediately work on AI, maybe your first internship, you can directly just do anything about it. It doesn’t have to be product management in AI because you’re fresh graduate. You’re just doing intern. So you can be a software engineer for AI startup. You can do anything for AI startup, do anything.

But we do have students, they are first, I have a sophomore students. Her first PM internship is working for NVIDIA to build those foundational models for NVIDIA as AI product manager. So you can directly go for AI internship, AI PM internship, the fresh graduate for short. The second time that people break into AI product management, our existing product managers.

So for those existing product managers, let’s say you’re working on IOT in the past or FinTech has nothing to do with AI. The easiest way to do it is you learn AI knowledge. There are so many different kinds of AI courses out there. And as we’re recording this podcast and Google also launches on AI courses out there.

So I encourage everybody, you’re already product manager. So just learn all the AI skills ahead of time, learn different kind of large language model, and do comparison to understand which one is good, how to make technical decisions around those. Those are the key steps all product managers today need to start to gain experience on.

And the third way for people to become an AI product manager is somebody who are currently have engineering experience by working in those AI companies, and I literally have so many AI engineers or data scientists inside of PM Accelerator. They do not know PM at all, but they have their own unique company advantage that build AI models before.

They probably are one of those, my own engineers years ago, when I build smart cities, they know how to train AI models, they know how I build my MVP because they’re part of the old process. They didn’t write requirements. Of course I wrote it, but those product management experience actually is easier to pick up compared with learning what is larger language model, how to train AI model.

So those engineers, I recommend you guys immediately learn anything about product management, learning how to go to market strategy, how to do customer interviews. This is going to help you to speed up your process significantly. And then final step, which someone has no engineering experience and no product management experience.

And for those people, I’ve highly recommend this kind of approach. Number one, study what is AI first. This is foundation. Number two, study what is product management. Number three, build your own hands on project. So building your own hands on project is a number one way for people with AI experience or without AI experience to show off your passion in the AI domain and also your knowledge. And also show them that you’re not just like talking and you have actually built several different kind of AI applications as a product manager. So I also want to differentiate this.

As a product manager we don’t need to learn how to code, but you can learn some simple ways, for example, I myself right now is building my own application on top of ChatGPT. And I do not know how to code at all. I got a PhD in material science nursery. Even today, I still don’t know how to code. But getting those hands on experience by using the API is a different plug in and also ChatGPT has instructions in terms of how to build applications on top of ChatGPT.

And all of this can be leveraged by people to gain hands on experience and learn, you turn it into a product portfolio, and then it’s much easier to convince the hybrid manager you can do it. So, therefore, there are different steps people can follow, and it will definitely help you to boost your career to make it much faster for sure.

Hannah Clark: Oh, those are really useful secrets.

So going back to some of the self-led courses and resources and other kinds of training courses that people can use to nourish that skill set, do you have any specific recommendations for resources or courses that folks can take?

Nancy Li: Yes, there are several courses I personally took and I really like.

And my approach is slightly different, not just taking courses, I also read articles. I also gain hands on experience, play with different tools. I give you some top choices, everyone can get started right away. So the number one resource is you can go to DeepLearning.AI. That’s the go to source by Andrew Ng.

And he was able to give several free resources on his website. But there is one course I highly recommend. Only takes one hour to take is going to significantly grow your understanding of AI and also hands on experience of AI, which is Prompt Engineering. So only takes one hour on the website is very fast, short course.

And number two, you can also go to Coursera and take those machine learning class and Coursera also work with different companies. They also work with IBM, work with DeepLearning.AI, work with many different company, create those machine learning classes. I recommend you guys also take the ones from DeepLearning.AI, they also have the machine learning courses you guys can study and which give you more end to end understanding of machine learning and you also need to learn like deep learning, what is deep learning. And I also recommend everybody to check out different articles published by NVIDIA. Again, I’m a big fan of NVIDIA.

I knew the CEO from like eight years ago. It’s crazy. So he’s very smart. Jason’s very, very smart. NVIDIA published lots of articles based on the existing fundamental models they built in house. And then other people like myself, my team was able to build application on top of his like existing framework.

And I literally want my best friend, I knew from like 12 years ago and still my best friend today, he’s working at the lead software engineer to create those kind of AI framework for NVIDIA. So those AI frameworks, something everyone need to learn in a high level. Because as I said, in the prior steps regarding learning AI product management life cycle, you don’t need to build your own, lots of time, making and leverage other people’s existing work.

So I recommend people check out NVIDIA’s website and where you can learn educational content and also learn existing framework you can use. And then the next part surprisingly is Google. Google also launches AI product called BlackRock, and they also adding many different applications on top of AWS today.

They also have very simple description of what is the AI framework and what kind of applications use cases can build on top of AI is all very simple, very short, and sometimes they have videos. You can use those resources as well. Of course, I have a full list of different kind of AI courses, and actually, I put it on my website drnancyli.com/aicourses with all lists of all the AI courses I have personally taken and evaluated, which I believe everybody needs to go take a look as well. You can just go to this website on drnancyli.com/aicourses and to get access to all those courses yourself and start learning for fun. So those are recommended AI courses to brush up your skills of AI.

And of course, inside of PM Accelerator, I have my own AI Prime Management course, which is more focused on what product managers need to do on a day to day basis what’s the end to end product management life cycle and also making important technical decisions, selecting different models as a product manager.

You guys can go to my website and learn more regarding product management courses as well. And of course, I also have my own version of AI product management course. It’s mainly specialized in product management for AI product managers and over there, we talk a lot about the end to end product management life cycle. And again, hands on experience and also building those real AI product with me together so that you have something you can show on your product portfolio. And over there, we’ll also give you the summary version of different kind of AI technical knowledge so that you can speak up the learning process because each of the AI courses out there takes two months to finish and every week it takes 10 hours.

So give you a shortcut to really tell you what exactly AI PM need to master as well.

Hannah Clark: So that covers the huge landscape of learning courses and resources that exist for folks who are interested in learning more about AI.

Now let’s talk a little bit about the employer landscape. First of all, do you know of any specific companies that are really in a push to hire PMs with AI specializations?

Nancy Li: Oh, yes, there are a lot. And every single week, there are so many new employers and new job opportunities coming up. And for example, there’s several exciting news. Amazon, they hire aggressively for AI PM, specifically Gen AI PM in some department of Amazon, which I cannot tell you which one. But inside the information, what I discovered was some departments of Amazon, they start to change the leadership, the VP left, the director left, and then hire a new leadership, and now the new leadership is very pro AI.

The old leadership was against AI, they want to use a traditional methodology to figure out do we really need to use AI or maybe we just do not jump on the AI trend right away. But the new AI leadership, the new product leadership really want to push on AI, so therefore what they did was in the company is they swap out the existing product manager who doesn’t really know much about AI and hire new AI PM, to put the old one on PIP (Performance Improvement Plan) and the new AI PM start to take over them like one by one slowly.

It’s very sad to see this happening, but you know what? It’s Amazon culture. They have this kind of like fast growing culture, so I will not be surprised to do that. But which also means that they literally hire a lot of AI PM to join Amazon right now for some department of Amazon. And of course Netflix is a trend.

They always hire AI PM. And last year, I think you guys should already read the news, Netflix paying people $900,000 per year in salary to join them as a senior AI PM to create those content generated by AI. And there’s also other very like a top AI startups, well you never heard of, but literally growing very fast.

One was startup, which I personally really like, which is Inkitt. I have a student interview with this company right now and their idea is very revolutionary. They are the readers of Netflix. So nowadays people read novels, any kind of reading clubs, and they can go to this company to read, but you’re able to engage with those characters in those books using AI.

Can you imagine you watched Game of Thrones or you read about Game of Thrones and you’re able to ask Daenerys questions live using AI and this characters are truly created by AI. So this company is already raising so much funding and it’s starting to become more and more popular and gaining lots of usage for the new AI product in the readers club right now.

So there’s more than one, there’s so many different AI startups and people need to pay close attention and I recommend everyone go to TechCrunch. We’ll talk a lot about those new tech startup ideas and new tech startup raising new fundings and all of those are great opportunity for you to discover new AI opportunities for sure.

Hannah Clark: Just as we’re starting to wrap up here, I know you’d mentioned that there is definitely some salary incentives for developing an AI specialization. So are there any other kinds of, you know, remuneration notes that you have for folks who are kind of contemplating, developing an AI skill set that might encourage them to take it seriously as the next move in their career?

Nancy Li: Yes, at the beginning, when I mentioned AI PM get paid more, which is true. I also want to share the other side of the salary and how you see salary from a different perspective. First of all, it is true. AI product manager get paid more, but I also want to highlight regarding the impact can make by creating AI, the efficiency you can create for day to day life and also creating some apps you personally like.

I, I really love to talk to some characters in Game of Thrones as an example, and on top of that, or also want people to really break down the salary. Because if you join some public trading companies, and I literally have student join, of course, all the fan companies and also even non fan companies, such as some company like Snowflake and those kind of tier of companies.

They have paid my student over $400,000 per year as AI PM. Now for startups, they may not give you the cash for $450,000. Those are cash, $440,000. Those are cash offer. But for some startup company, they probably give you salary base. So for example, $160,000 at the base, but give you lots of equity. But think about how much the equity is going to become the next million dollars, we never know.

So therefore, I advise everybody really choose the startup very wisely. You need to align with the mission of the company, creating the right impact. And also need to pick the right horse. Because you never know, maybe you join the next Google. Then you will be the one speaking on this podcast next time and also join my own podcast. I will be inviting you guys regarding which AI product to pick for sure. So be wise about your next career move. Do not just jump into AI space right away. Pick the right horse.

Hannah Clark: Well, with that, thank you so much again for joining us, Dr. Nancy Li. And I know you’ve mentioned a few of the resources that people can find on your website.

Is there anywhere else that people can follow your work online?

Nancy Li: Yeah. You guys can go to my podcast, Product Insider podcast by Dr. Nancy Li. My YouTube channel, just search Dr. Nancy Li, director of product and also DM me on LinkedIn. I’m very active on LinkedIn. I respond to all the messages when people DM me and yeah, go to LinkedIn and also search Dr. Nancy Li. That’s where you can find my social presence. Also, you can go to my website, drnancyli.com/aicourses, check out those free resources I have collected for all of you guys.

Hannah Clark: Fantastic. Well, thank you so much for joining us again. This has been so interesting to learn. It’s such an exciting area for folks to be aware of.

Nancy Li: Thank you for having me on the show, Hannah, and let’s keep in touch. Everybody, let’s jump on the era of AI together.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The CPO Club wherever you get your podcasts.