Duolingo’nun Ürün Direktörü Ürünleri Paraya Dönüştürme Konusunda Ustalık Dersi Veriyor

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: I think we can all agree that our decisions are fundamentally about how we can make our products valuable for our customers. But today we’re going to focus a little deeper on the ‘valuable’ part… as in, how can our products make money. We know we need to be profitable, but the road between our product and a healthy ARR is not exactly a straight line. Getting it right is obviously mission critical, but there are a lot of different ways to get there—and that’s exactly why this conversation is so interesting.

My guest today is Cem Kansu, the head of product at Duolingo, and I couldn’t have asked Santa for a more perfect case study of a high value product with a super smart approach to monetization. You’re about to learn what has and hasn’t worked for Duolingo over the years, find out what the Premium Trap is and how to stay out of it, and hear how the introduction of Gen AI has factored into their pricing strategy today. Guys, this conversation was so cool, so engaging, and so packed with real lessons, it’s basically Duolingo for product monetization. Let’s jump in.

Welcome back to The CPO Club podcast, everybody. I am here today with Cem Kansu. We’re so honored to have him here with us today.

Cem, thanks for joining us.

Cem Kansu: Thank you for having me. It’s great to be here.

Hannah Clark: Yeah. So can you tell us a little bit about your background and how you got to where you are today at Duolingo?

Cem Kansu: Sure. So I joined Duolingo eight and a half years ago, which feels insane for me to say out loud, but time does fly when you’re having fun. So I joined Duolingo in 2016 and I worked on monetization for about the first five, five and a half years of my tenure. And monetization for us is I’ve worked on building out our business lines that include ads, in-app purchases, and subscriptions. And for the last three years, I’ve been head of product here at Duolingo.

I lead the product management team and we have about 40 PMs in the company. And a lot of product bets that I ended up spending a lot of my day on. And then before Duolingo, I was in business school. Even before then I was at Google for about four years. And then if you go even before then I’m originally from Turkey and I did some part of my schooling there.

Hannah Clark: Wow. You did Google before you went to university?

Cem Kansu: No. Before grad school, before business school.

Hannah Clark: Okay. I was thinking like, geez, it must’ve been quite the prodigy.

In any case, this is a great segue, cause today we’re going to be talking about monetization, which is something that we talk about, shockingly, not enough on this show. We talk a lot about development. We talked a lot about research. We don’t talk a lot about how to make money.

So let’s start off with kind of a general question. So what do you see as the biggest monetization challenge that product teams right now are facing in the market? Representing Duolingo, how have you guys navigated this?

Cem Kansu: It’s a great question. I think one is figuring out the best model to monetize your product. I think different if your app is catering towards businesses, you’ll take a different approach than consumer. I can speak to a lot more consumer. That’s where I’ve spent a lot of my career on, actually.

But thinking about consumer monetization, it used to be the case that I think in the early social media app craze days, like the Facebook days, the earliest wisdom was always, get users, you can figure out how to monetize later. Because if you can reach a massive user base, you can figure something out. Out was the wisdom. I think the wisdom has generally shifted now to you must show how you can monetize earlier in your life cycle, monetize and grow at the same time as more of the common advice that you find for startups these days, especially consumer startups.

The Duolingo journey was quite interesting, I would say. So Duolingo started in 2011. Those were still the days where investors or like wise people would tell you just get users. You’ll figure it out later. And I think Duolingo for the most part followed that philosophy of let’s grow our user base, find product market fit, and we can figure out how to monetize later.

Duolingo itself also had a monetization plan that was really elaborate. Our founder, CEO, Luis was the founder of CAPTCHA. I don’t know if anybody here knows CAPTCHA, but it’s the squiggly letters that you put on your website. But the idea with CAPTCHA was you would actually also crowdsource digitization of books.

So every squiggly letter you type would be crowdsourced into actually getting scanned book images into actually recognized digital words. So anyway, Luis had pioneered this kind of like crowdsourced book digitization model with CAPTCHA. His idea with Duolingo would be something similar where as users learn the language, as they did Spanish exercises, let’s say, they’d be translating English articles into Spanish piece by piece.

And then Duolingo would sell a translation service to whoever needed it. The problem with that business model was, one, turns out if you’re just learning a language, your translation quality is just bad. So you generally end up needing either a third party or another service to uplevel translation quality.

The other one is, you end up with the, translation is a race to the bottom business model, where you’re always competing for a lower price. You do it for, let’s say $2 an article, someone in the Philippines does it for a dollar an article, and then it’s always a fight to the bottom. And now in hindsight, that would have been a pretty bad business model with LLMs.

I think translation has become almost instant at this point as well. So anyway, long story short, when I joined, Duolingo had abandoned this business model and was working on what could be next. We worked on freemium as what fits into what our product is doing, which for us made a lot of sense both because of our mission and what our product stood for, which is access to education.

We never wanted to lock all of our educational value behind the paywall. And to your earlier question, I think this is what apps end up facing quite a bit, which is how much to provide for free and how much to put behind a payment, whether that’s a one time payment, whether that’s a recurring payment, like a subscription, we have decided for both of our ethos and where our company mission was.

And for the long term growth of Duolingo, that the educational values should be completely free. So the, every French content you can access is the same, whether you’re paying or whether you’re free. However, the added bells and whistles like removing ads or getting access to more lives as you go through lessons would be some of the paid features.

That’s what we started with and we’ve stuck to for the past eight years as our business model. And I think again, to your original question, what do apps face? I think that’s one of the key decisions you end up making, which is should you offer some part of your app for free, which parts? And I think the underlying variable to meet there is what you offer for free should eventually help you grow.

So for Duolingo, when we offer the most valuable part of the app, which is the fact that you can learn a language or learn math music for free, it really helps build an organic flywheel because if you use it and you like it, you tell your friends. If everything is paid most people don’t even get access to it and don’t tell their friends. So the organic flywheel doesn’t build. If you build a complete premium product, then you end up having to build a paid marketing machine because the product needs to grow and you end up having to spend a lot in marketing to grow, which is not necessarily a bad thing, but it’s an entirely different business model.

Hannah Clark: I think that so much of the conversation around pricing monetization comes down to this very delicate balance on so many different levels.

But I want to talk a little bit about the Premium Trap since we’re talking about, premium and, trying to find that balance.

First of all, can you walk us through that concept of the Premium Trap first and how has Duolingo managed to sort of avoid it while still growing user base and revenue?

Cem Kansu: Definitely. The Premium Trap, I think, is, I don’t know if this is a common term, but we use that at Duolingo a lot to at least describe the dynamics of what a lot of, honestly, education apps fall into. And I’ll take Duolingo as an example here because it’s easier for me to talk about. So we’ve existed for 11, almost 12 years now as a product.

Over the 12 years, we’ve seen a lot of competitor products come and try to do exactly what Duolingo is doing with one key difference, which is make the whole app premium. And this generally goes as, someone starts doing something very similar to Duolingo. They teach a topic, let’s say a language or math or music.

There’s a lot of products that are educational out there. They either start fully paid or they start free, but then it’s quite hard to build an organic flywheel. So very quickly, these apps land on the insight that if we make it fully paid, we can monetize. And if we monetize, we can spend on marketing and build like a loop where if your LTV is higher than cost of acquisition, we can build a flywheel where you spend on marketing, you acquire subscribers and use that money to spend more in marketing. This is mathematically a sane strategy. You can do this, but there’s two things that happen to companies that generally take on this business model. So that, and I think that’s the trap part.

One is you start spending a lot of your efforts, all your growth effectively comes from paid marketing. You start spending a lot of your efforts, whether that’s your time, whether that’s your leadership bandwidth or whether that’s your actual cash in the bank on marketing rather than building an excellent product.

So your entire orientation as a company is more in marketing and acquiring users than building excellent products. And with education, the really hard part that we have learned at least is the hardest part is building an engaging experience and taking an educational topic and making it exciting, fun, and fun to use.

So if you’re not spending a lot of effort there, you’re certainly falling behind on what everyone else is doing or what, where education is supposed to go. So that’s trap number one. Trap number two is once you’ve gotten on this flywheel, let’s say you’re building a revenue base that is fueled by paid marketing.

It’s almost impossible to step out. Meaning you can’t easily say, Hey, let’s actually move some of our key features that we know people pay for and make it free so that will help us also build an organic flywheel. Because then it slows down your growth and living through a slow growth period to transition from like heavy paid to free is, I guess I’d never seen it.

It’s really hard to do. The reverse is a lot more common where a lot of apps start fully free or freemium. Then they realize they’re not patient or they want to grow more. They start investing in paid marketing, but paid marketing has to pay off with payers because that’s the positive revenue flywheel.

Anyway, so that’s part of the premium trap where if you create a premium app and you are your only growth is coming from paid marketing, it’s very hard to get out of that trap and build a scalable growth model that’s fueled by organic at the same time. And we’ve at least our view has been organic growth is certainly hard to control because you’re not controlling with spend, but it’s also obviously super cheap because it’s technically free.

So we’ve bet on making features that are free and excellent so that our users always tell their friends and that creates this kind of organic flywheel.

Hannah Clark: Yeah, I tend to be aligned with that approach as well. If you think about the product as a party. If you’re throwing an amazing party, people will call their friends and say, you’ve got to get down here.

But if you spend a ton of money putting posters around saying this party is great, but then there’s no food and, the music is lame. You’re inviting people where there’s effectively nothing happening. And so the spend ends up being for not, right?

Cem Kansu: Yes.

Hannah Clark: To just throw an analogy out there. But yeah, I’m fully aligned with you guys approach. And I think that’s a really great insight.

To go a little bit deeper than on the freemium model, I think that’s part of what has made Duolingo so successful is you guys have really nailed that kind of balance in the freemium space. You guys really understand your user base who has created a really compelling product.

Just about everyone that I know who is a Duolingo user has said that it’s really the only language learning app that they’ve actually stuck to. I’m not even being paid to say that. It’s really true. So what strategies have you guys found to be most effective in maintaining that free access and making sure that people are able to access all that value for free while encouraging users to upgrade to premium?

Cem Kansu: A few things. Let me see. The one is how you set guidance and guardrails for the entire company, which has always been a free user and a paid user should be able to learn the same amount of French period. Meaning, a free user should not have a different set of content than the paid user content should not be the differentiator for payment.

That’s just a view we had because we’ve always realized if you provide an extensive, great educational experience for free users, either they’ll tell their friends, it helps them grow, or they’ll stick with the product for much longer periods of time, and maybe three years down the line, they can decide to upgrade to the subscription.

So it creates longer term habits, much easier. That’s been I guess the guiding principle is one strategy. Like you guide your product direction in a way where there’s like a sacred cow, which is do not put educational content behind the paywall. And I think we’ve stuck to that and that’s helped us build a quite large user base from the beginning.

So that’s one. The other one is you use A/B testing and data to your advantage to decide what kind of features help growth and what kind of features help monetization. If there’s a clear divide where let’s say, I’m going to pick a feature that we have. Leaderboards is a feature that we have. Leaderboards, you can, let’s say you’re at the invention point of the feature where you’re going to decide, should this be a paid feature?

It could be a paid feature. It’s not really educational content. It fits with your ethos. Or it could be a free feature, but what benefits more? If you make it a free feature, does it help user growth more? Or if you make it a paid feature, does it help revenue more? I think you can one, use your best judgment, or if not, you can test and see what happens.

For us, obviously one view has been social features like leaderboards, where you get to interact with other users, benefit a lot more from a higher volume of users so we made it available for everybody. For niche features that we have a custom practice feature, for example, it made more sense for us to monetize that because it’s very niche.

Most users don’t want it, don’t need it, but the ones that do are willing to pay for it. And if you don’t do it, you don’t lose out on any education value. We call this the practice hub. We’ve decided to make it a paid feature. So long story short, I think deciding if you’re adding value to your product with a new feature, what metrics drive the most from that feature. And if it’s growth, then definitely don’t monetize it because then you want your user base to grow with it.

I would say maybe the third one is I’ve talked about the premium trap a little bit. I think aligning everyone in the company, what your strategy is, goes a long way. Meaning, are we a premium product where anything is free to monetize or are there sacred cows that should never be monetized? I think it goes back to my first point, but it really helps. Especially in a growing organization, if you align everyone where they should experiment? Can you experiment with making the whole app paid or is that an experiment you should never run?

So I think aligning on the lack of better terminology, like what’s sacred and what’s not, I think also really helps.

Hannah Clark: Yeah. I think that’s really important for establishing like a shared ethos as well. And I’m glad that you brought up A/B testing, cause I am curious about how that sort of fits into your approach.

Can you share some insights or some discoveries that you guys have gleaned from your A/B testing around monetization and retention?

Cem Kansu: Sure. We run a lot of A/B tests. I think over the span of a quarter, I think now we run more than 300 A/B tests on Duolingo just to give a sense, which most folks, when they open up the app, they don’t realize. It’s, it feels, it’s designed to look very simple and it is designed to be very simple, but we run a lot of A/B tests.

And these range from maybe changing the shade of the blue on the button that you tap on to be a little lighter, or something as drastic as like a massive redesign of our entire homepage. So anything counts as an A/B test. Every change we do goes out an A/B test because data can tell what the change is doing better than our intuition.

What insights or discoveries have we had? This is a very long list, so I’m going to maybe do a brain dump of an answer of various things we have seen. One of the things we’ve seen, especially being a product that balances education with engagement, there is a sweet spot in how hard or how easy you can make a lesson.

Meaning, turns out we’ve tried many different extremes. We’ve tried making the content easier and easier, and we’ve tried making the content harder and harder. On average, users want to get 80 percent of the questions right. Turns out that’s roughly the sweet spot. If someone gets a hundred percent of the answers right, they churn because it’s, they think it’s too easy for them.

If they get 30 percent of the questions right, they feel like they’re not doing a good job or the content or the app is not for them. So there’s a sweet spot. It’s 80 percent is not exact, but we’ve learned that range holds well for balancing retention and building a habit. So learners come back the next day and keeping users engaged.

What other insights have we learned? It makes a massive difference if you’re trying to upsell folks to a paid product, like for us, we have two subscription products called Super Duolingo and Duolingo Max on how you talk about it, how you present it. A lot of subscription purchases, especially in freemium products are emotional purchases.

So there’s a lot of testing you can do to uncover what works and getting a user to be convinced to buy your product. I mean, this should probably be no surprise, but like optimizing how you present a subscription upgrade to a user. We have been working on it for, I would say now eight years. That’s been how long it’s been since we launched our subscription and we’re still finding a lot of new ways to get users to upgrade.

For example, it is a very good idea to give users a try and buy approach where you give someone to try the subscription for three days and then ask them to start a free trial that works much better than just saying, start a free trial. Or on the purchase button, if you say try free for seven days, that does some performance.

But if you say try for $0, weirdly try for $0 does much better. If you’re giving an offer, if you write last chance, if there’s an actual last chance happening where the offer is about to disappear, the click through rate goes up a ton when you write last chance, instead of not. There’s a much longer list of these things that we’ve tested and learned, but I think my main takeaway is there’s a lot to do to figure out what works in your product for upgrades into subscription products.

I guess the last learning we’ve had, and we’re trying to navigate what this really means for us, honestly, because there is certainly something as A/B testing too much and putting yourself into a local maximum trap, where when you have really great tools and like every product team is really good at running A/B test.

One trap you can easily fall into, and sometimes we do this to ourselves, which is, you start testing only incremental, tiny changes. And even the number that I say sounds impressive, where you can say, Oh, we ran a hundred A/B tests. But if all you tested is like the button color, the pixel by pixel, very small things, then you’re never testing what a complete different imagined version of that feature could be.

And teams sometimes do get incentivized because, they get some wins. They run a lot of A/B tests and it feels good at high volume. But maybe actually there’s, if you step back and say, what if we remove this entire screen and did something else entirely turns out sometimes that’s two X better, but you don’t never get there because you’re always incrementally testing.

So avoiding the incremental testing trap is a learning that I don’t think we’ve figured out exactly how to act on, but when we see it, we’re like, Oh, we’ve A/B tested way too much into kind of a local maximum. So maybe that’s the third one I’ll leave with.

Hannah Clark: This is an interesting point. We’ve talked a little bit in a panel earlier about what done looks like to a junior, to an intern versus what done looks like to a senior product manager, one will see something as just completed, whereas the other one is looking for, insights or what kind of value or impact is, is done generating.

So I see it as sort of analogous to that, in which we’re are we running A/B tests for the sake of saying we ran 300 A/B tests this year? Or are we running A/B tests in order to get, what are some of the risks that we can try and take and get like a better sense of where the temperature is from the user rather than just fine tuning things that, there’s diminishing returns.

Cem Kansu: Exactly. And I think the other example is it better to run 300 A/B tests and get, let’s say a few percentage point improvements? Or run one big test that might actually be a 50 percent improvement? And it’s hard to decide in the moment. In hindsight, you can easily say 50 percent will take it, but in the moment, it feels safer to run the 300 and get a few percentage points.

But again, like you said, diminishing returns becomes the problem.

Hannah Clark: Yeah. Yeah. I mean, each of those tests, even if they, there’s a learning from them, it’s it’s still going to cost you money to run the test, so.

Cem Kansu: Yes.

Hannah Clark: Those were really valuable. I really appreciate you sharing those

Web designers, this one’s for you. I’ve got 30 seconds to tell you about Wix Studio, the web platform for agencies and enterprises. So here are four things you can do in 30 seconds or less on studio. Adapt your designs for every device with responsive AI. Reuse assets like templates, widgets, sections, and design libraries across sites and share them with your team. Add no-code animations and gradient backgrounds right in the editor. And export your designs from Figma to Wix Studio in just a click. Time’s up, but the list keeps going. Step into Wix Studio and see for yourself.

So let’s shift a little bit over to talking a little bit about pricing strategy.

So you talked a little bit how things have evolved over time. When we’re digging into the current iteration of the pricing strategy and, you’ve got like different paid tiers, you’ve got your free tier. What are some of the factors that have influenced how features are priced today, or your kind of the differentiation between the different paid products that you offer?

Cem Kansu: Our pricing strategy has evolved mainly through package evolution, I would say. So Duolingo has roughly always had for the past eight years has had a subscription product. But what has happened over time is one, our first launch was, first, it was called Plus Duolingo, and then we eventually rebranded to Super Duolingo.

Sounded like a better name. We also did a big redesign of the product and also we launched a family plan package. So that came at a different price point. So that was almost 50 percent higher than buying it individually. That was a good deal for group plans. That was our first evolution. And then we also launched our higher tier, which is Duolingo Max.

That was about two years ago now. And that is almost double the price of Super Duolingo. You get added features that are using a lot of generative AI to give you conversation practice. And that’s been, I would say that the second phase of our evolution, because that certainly changes your price and adoption dynamics, having different packages for different folks.

Like higher propensity to pay and higher ability to pay might now go to Duolingo Max versus a group plan might go to family plan. We also just recently load on a student plan. So we’ve become quite segmented in the offerings we have. And the way we got to the right price points, honestly, was mainly through A/B testing.

What I would say we have learned, we’ve tried many different things to front run, not having to A/B test prices, cause A/B testing prices is a tricky ball game. Where, even if you have the full ability to run A/B test, which we technically do, for example, we don’t have contractual agreements with like third party content that our prices might be a certain level, et cetera.

So we can technically offer anything at any price we want. However, it has a cost to run A/B tests. So we’ve tried, for example, asking users, doing surveys. We’ve done, try to do qualitative, quantitative research. We’ve tried doing what you would call a painted door tests where you get people to click on a certain price, but there’s nothing to buy on the other side.

You say, Hey, this product is actually not live yet. None of these, I would say, have given us great signal on price, unless you’re looking at super divergent prices between, oh, should this be $5 or $100? Then maybe you could get like rough signal. But if you’re trying to test, should our annual subscription be $80 or $85 or $90?

We have never been able to get good signal except outside of A/B testing. So we’ve eventually A/B tested all of our prices to what it is now. And that has led to our prices being what it is. Now going forward, I think this approach will probably hold true. It turns out, I think purchase behavior for a premium app really depends on how it’s presented in the app and what the moment of purchase looks like, what did the user experience just before when they hit the purchase experience.

That’s why we’ve found it really hard to replicate the same results that we see in A/B testing in research, because research tends to be, Hey, would you buy these features at this price point? And a lot of our purchases, I alluded at this a little bit, are emotional. It’s you, for example, went through a lesson. You’re really excited. You got the perfect lesson on Duolingo. You might just say, you know what? I love this app. I’m just going to buy their subscription versus a survey can’t really replicate that. And I think the pricing dynamics, that’s why we’ve seen A/B testing as the main method to determine pricing.

Hannah Clark: This is a really important takeaway. Because when we’re talking about the kinds of insights that you can glean from purchase behavior, it’s so divorced from the headspace that users are in when you’re interviewing them. Because people during interviews, they’re wanting to present their best self.

They’re thinking with their rational mind of, what I should I do? It’s very different from the purchase behavior in the moment when you actually, have that emotional momentum to actually, hit the subscribe button, so I can really appreciate that for sure.

But I do want to talk a little bit more about AI because you mentioned that you’re currently using generative AI features as a conversation practice tool, which I think is really cool. It’s a really innovative way to use generative AI. How do you balance the cost of an innovation like that with setting a price that users find reasonable for Duolingo Max?

Cem Kansu: So, for Duolingo Max, the feature, the main feature of Duolingo Max is what we call Video Call with Lily. Lily is one of our characters.

We have multiple characters that appear throughout your Duolingo lessons. A Video Call with Lily is exactly what the name implies. You call Lily to practice Spanish or any other language that you might be learning. The one huge benefit is we tailor the entire level of Spanish that she is talking to you to your Spanish level.

Since we know every word you have learned on Duolingo, she will tend to use those words at least 89 percent of the time. She’s going to use those words. 10 percent of the time, she’s going to introduce some new words, but she’s going to speak to you at your Spanish level. So it becomes this like perfect conversation practice.

The adoption has been really good. So this is, we’re going to continue investing a lot in this kind of interactive conversation features for Duolingo Max that use generative AI. Now, to price these features, one new dynamic for us with Generative AI is there’s a cost because making calls to LLMs today is quite costly.

So we factor in some average cost based on average usage because we’re selling a subscription product that users pay for either a month or a year. And we have to figure out, is the unit economics solid or not? So we estimate what the unit economics of cost is going to be based on what we end up paying kind of these LLM calls.

And we price obviously above that. So at least we’re not unit economics negative. And then beyond deciding what the right price point should be, obviously it should be above cost to our best ability on our cost estimates. And then we try to figure out where the demand might be. So I think A/B testing really helps figure out where the demand might be.

And demand for us is also anchored on Super. So since we’re offering Max as more of the AI heavy feature set and Super as our anchor subscription. Should it be double the price of Super? Is that where the demand really is? Should it be 50 percent higher? So then we A/B test where we can see where the highest demand is and then determine the final price.

Hannah Clark: So we talked a lot about how presenting certain features can really impact how folks perceive the value of things and willingness to pay and, using that kind of emotional trigger to help people through the funnel.

What kind of language do you guys use around AI and have you noticed any kind of, or maybe you’ve done some A/B testing around specific ways to position an AI feature in order to, encourage people to upgrade their subscription?

Cem Kansu: Yes, we’ve done quite a bit of research to try to figure out how to communicate the feature value and there’s many different ways you could do that. For example, for our feature, the video call with Lily, is it better to talk about Lily? Is it better to talk about conversation practice? Is it better to mention, Hey, this is state of the art generative AI.

So there’s many ways to present the same feature, either, both actually through research and A/B testing, AI messaging seem to resonate the least. When we label AI as like, Hey, like there’s like new AI thing, even though there’s a lot of hype, I think we are reminded of the reality that like, yes, in tech circles and product management circles, AI might be a really attractive tagline, but that’s not really the case if you look at the entire user base of Duolingo, which is very global.

Only 20 percent of our users are in the U.S., 80 percent are abroad, and tech literacy is clearly not as much as people who work in tech. So the word AI doesn’t necessarily ring the same necessarily strong bell that it might to tech audiences. So one learning is we don’t actually label most things as Hey, this is AI. Because every time we’ve tested it, we’ve done research on it, the user response was, I don’t particularly care. What they do care, this will sound obvious in hindsight, is what value will they get if they pay? And for us, that’s been, you can learn how to speak better.

You can improve your conversation skills if we use that as a way to describe the feature, it resonates a lot more. I guess this is super obvious in hindsight, it wasn’t to us at first, but that’s the marketing line that works really well, which is just talking about what value users like what’s in it for them, what user value will they get if they pay resonates much better.

Hannah Clark: I have a theory around this because I agree that there’s a certain level of tech literacy that’s lacking in some user bases. And then the other thing also is I think that there is a bit of a sense of AI fatigue in which the use of AI feature, I think a lot of folks are just used to seeing AI features pop up in, products that they’ve used for so long and they often will fail to ascribe any kind of added value to a lot of them because not all AI features necessarily present a better user experience.

I think that this is a very smart takeaway in general to be leading with what’s the value of a feature because often I think that’s the missing piece in which people are saying AI, I see that everywhere. What’s it in it for me? Is it actually going to improve my experience of this product?

Cem Kansu: That’s exactly what we’ve seen.

I think there is even some negative connotation attached to AI. Either from products that have added AI features that don’t really do anything. There’s a lot of, imagine the number of dumb chatbots you’ve spoken to that are labeled AI. Or some people even perceive it as negative. It’s is AI going to take over the world?

What is it going to do to me? So it’s not always even perceived positive, but I think even beyond that, what we’ve seen is, again, it’s like the direct link between if I pay, what will I get? I think if you can easily convey that tends to work better.

Hannah Clark: I think that’s a very smart approach. And I obviously, this is just my feelings. I’m glad that you guys have the data to validate that.

So we’ll talk a little bit now, but what’s next for Duolingo? Cause it’s very exciting time for you. You’re expanding into broader markets, like math and music. Wish that I’d been around when I was in high school. So what criteria are you currently using to evaluate whether a new product aligns with your mission and competencies given where you are currently as a product?

Cem Kansu: Yes, so we are now teaching math and music, and the reason we have decided to expand to math and music is, let me start with, I guess, competencies, because I think that’s fairly easier. What Duolingo has done is effectively make self learning fun. We’ve added many things that games might do for entertainment reasons or other apps, but it made them work for education.

We’ve added the streak, we’ve added the leaderboard, we’ve added quest systems, we’ve made lessons really bite sized and fun. None of these are language specific, so they can actually be applied to many different topics. Infinite number of topics could use more fun. However, the reason we picked math and music is two fold.

Actually, the reasons for both are actually quite different. But one is, is the market big enough? Meaning if we do apply everything we’ve learned and help teach these topics really well, is there enough demand in the world for the return on investment to make sense? And for math and music, the answer is yes.

If you look at the number of people learning math in the world and learning music in the world, or wanting to learn math or wanting to learn music, it’s quite high. So the market is sizable. That’s one. But the other one is the criteria we use to decide math and music have varied from one topic to the other.

So music already is very similar to language. If you survey the general population and say, do you wish you learned more music? Or do you think you want to learn more music? Similar to language, a lot of people in the world have this demand in their heads of I wish I learned more Spanish. I wish I learned how to learn the piano, but it’s not easily accessible.

It takes many hours to do these things like becoming fluent in the language, learning to play the piano, but there’s certainly demand to do it. It’s just, there’s no easy way to do it. Music is very similar to language that we believed we can make it easy and fun. And that would be we could tackle music that way.

Math is a little different. I think if you survey the general population, I don’t think a lot of people would say, Oh, I wish I was doing math right now. I think instead, like humans have math anxiety instead. However, very similar to language, especially thinking about people who learn English, math has a real shot at changing people’s life outcome.

If you learn math, then you can understand maybe how interest rates work. Like you can understand maybe how finance works and you can either apply to your own or get a better job, etc. Learning English is very similar. Learning English gets you to go to a better school. Learning English gets you to move to a different country, etc.

It unlocks opportunities similar to math. So it really fit into Duolingo. The other thing is we believed with math, we can do what we did to language, which is expand the market itself. A lot of people on Duolingo, actually roughly 80 percent when we run surveys, this is what kind of what we get when we ask them, what tool were you using to learn the language before you joined Duolingo?

Majority of the answers we get are, I wasn’t actually learning a language before Duolingo. So turns out a lot of folks with Duolingo starting learning a language. And now with our insane social media marketing, there’s a lot of stories where, oh, I thought you guys were just a funny owl. Realized it’s a pretty good app.

So now I’m learning a language. Our TikTok account has our mascot duo doing really unhinged, funny stuff. And a lot of people discovered Duolingo through their first entries, our TikTok videos. So anyway, long story short, math, we believe we can expand the market and get more people in the world to be excited about learning math, similar to how we expanded the language learning market.

And music was more making it easy and capturing the demand that already exists. That’s been our guiding light for these two topics. That being said, we’re still very early. We believe there’s a lot more to do on both of these topics. So we’re going to be working a lot more on both of them.

Hannah Clark: I’m surprised too, that math is happening at the same time as this expansion to music. Because I agree, I think music and language are very similar. I consider music to be a bit of a language in which you learn the rules in order to know how to break them in more of a creative context. Whereas where I see math as a collection of different hard skills that can be applied in different contexts.

So I’m very curious how you guys, first of all, how you’re approaching the challenge of looking at math and teaching math using that same kind of research, or I don’t know if you’re applying kind of some of those same learnings, or if there’s a whole other way of approaching product development, almost like you’re coming up with a whole new product line.

Like, how are you guys looking at the development of the math product?

Cem Kansu: There are similarities and there’s differences. A lot of what we’ve learned on making lessons fun or applying to how we’re building math lessons, they should be bite sized, they should be fun, they should have multiple interactions.

You should never passively read content. You should always be answering questions. This show would be interactive. All the principles, anything we’ve learned to build a lesson, applies quite well to math. What I think is different, actually, is how we build content. So we are building content that teaches generally K12 math, but we’re trying to make, take that content because there’s that content already exists in the world.

You can pick up a math textbook and it’s there, but we’re trying to take that content and make it a lot more real world and a lot more fun. And I think that’s where a lot of new innovation is coming to play where, for example, instead of having you just calculate two times three, we’re giving you maybe like a slight puzzle to solve where one of our characters is I just bought two bags of apples.

Each of them are three kilos. How much am I going to carry you now? So again, not that different. Actually, the math is the same, but we’re trying to add this real world layer and making a lot more fun and applicable for the general population. It’s a bet. We’ll see if it works, but for now that’s been where the big innovation is, which is taking textbook content, but turning it into an a lot more fun format that we have created.

And that’s been where a lot of our work has been going into.

Hannah Clark: Oh, that’s really fascinating. And I really hope it works out for you folks, because that is definitely a problem. I can speak firsthand that definitely doesn’t have a lot of great solutions right now.

We’re almost out of time. But just really quickly, I did want to touch a little bit on some trends and challenges in edtech since we’re on the topic.

What do you think will be shaping the next phase of growth for Duolingo as far as trends in the world and kind of the industry at large?

Cem Kansu: I think for us, we already talked about actually the two things that we believe will be transformational to us. One of them is Duolingo is still very much known as the language app.

But we want to go from being the language app to the language app, the math app, the music app, at least. And so how can we succeed in becoming a broader where you learn all these topics? We’re early, we’re making a lot of progress, but we certainly looking three years ahead, that’s where we would like to take Duolingo.

The other one is LLMs have really unlocked this massive use case for us, which is being able to practice language. Another use case that’s unlocked for us is content creation. Now we use LLMs for a lot of our course content, which now gives us a lot of speed if we’re creating new language courses. So how do we capitalize on LLMs to make Duolingo more and more efficient, build more interactive features that use LLMs and make the whole company more efficient?

These are, I would say, just, you asked about trends. I think the biggest trends that we are betting on, one is expansion beyond language. The other one is making sure we can capitalize LLMs for features, processes, content creation, everything that helps Duolingo operate better.

And then the third one I would say is advanced English Learners. Advanced English Learners, if you look at language, is a very specific cohort, which is actually the largest language learning cohort in the world. However, Duolingo’s product offering has been weak for Advanced English Learners. With Video Call, with a lot of new content, we’re trying to improve Duolingo.

Hopefully we make progress and succeed there. But these three, I would say, are our largest product bets.

Hannah Clark: Very cool and very exciting as well. I think that a lot of folks are tuning in because they’re so in love with Duolingo as a brand and as a product. So I think we’re all rooting for you guys. Not that you really need our help.

But thanks so much for joining us, Cem. Where can people follow your work online?

Cem Kansu: I’m on LinkedIn and Twitter. My handle is my first name and last name. So it’s Cem and then last name is Kansu.

Hannah Clark: Awesome. Thank you so much for joining us. This has been such a great conversation.

Cem Kansu: Thank you, Hannah. Thank you for having me.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The CPO Club, wherever you get your podcasts.