The AI-Enhanced Team: A New Playbook for Product Leaders

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: One of the many disorienting things about this AI revolution is that we are collectively inventing a new reality, which I probably don’t have to tell you is much less utopian than it sounds. But what’s also interesting is how we’re watching in real time as both leaders and ICs are, I dare say, struggling to navigate the messiness that comes along with forging a new path.

The questions around things like workplace ethics, output quality, and impact on future outcomes are on the minds of everyone in the org chart. But that also means we have a great opportunity to learn from each other’s breakthroughs and design our own decisions around the reality we want to end up with.

My guest today is Andrew Saxe, VP of Product at Smartling. Smartling falls into the translation and language services industry, which is practically a poster child for industries impacted by AI. When Andrew started at Smartling, their team was performing translation services manually, and AI was just an old sci-fi movie featuring Haley Joel Osment as robot Pinocchio.

Needless to say, things have changed. In this episode, you’ll hear Andrew’s take on separating genuine AI value from inflated hype, ethical considerations leaders need to be thinking about when deploying AI-powered workflows, and why the skills in highest demand have shifted faster than anyone expected. Let’s jump in.

Oh, by the way, we hold conversations like this every week, so if this sounds interesting to you, why not subscribe? Okay, now let’s jump in.

Welcome back to the Product Manager podcast. I’m here today with Andrew Saxe. He’s the VP of Product at Smartling.

Andrew, thank you so much for making time in your busy schedule to be with us today.

Andrew Saxe: Absolutely. Great to be here.

Hannah Clark: Can you share a little bit about your background and how you arrived at where you are today?

Andrew Saxe: Yeah. Do you know what I’ve always had an interest in probably the mid nineties. I had an interest in internet and it was new and I’ve always been in the world of technology and studied a lot of technology and all of my jobs have been in development and product.

And I was a fake web developer for a while, making not the best websites or code, and eventually moved into product. And I’ve been at Smartling about 14 years, I think 15 years in July, so quite a while. But yeah, always pet up product Smartling.

Hannah Clark: Yeah. Cool. Yeah, 15 years is, it’s no nothing to sneeze at.

Andrew Saxe: It’s not a little bit of time. That is for sure.

Hannah Clark: Especially in the tech world. It’s dog years.

Andrew Saxe: Yeah. No, it’s pretty, it’s like my, it’s my longest relationship, I think.

Hannah Clark: So today we’re gonna be focusing on the evolving ethics around AI from a leadership lens. And so to kick us off, did you wanna frame the conversation through your observations at Smartling over the past 15 years of your tenure?

So during your time, obviously you’ve seen the, you know, Smartling is in the translation industry, so obviously there’s been a lot of big technological transitions that have happened since you started the company. So there’s cloud adoption, transparency, and now we’re in the age of AI. So what is different about this shift?

Andrew Saxe: Yeah, absolutely. So yeah, so I’m in the harlems in the translation business, so I can turn English into French and Spanish and that type of thing. And when you think about that process, it’s really been a lot of people and like translating things as centuries old. It’s always been there turning languages into different languages and it’s always been really a people focused endeavor.

And thinking about the other technological advances over the last 15 years, and obviously cloud was probably the biggest one. They were really like business and organizational impacts, and this shift to AI really has already had a profound impact on the translation world and probably elsewhere really is gonna start impacting humans more and more.

And that like the, although cloud was a huge impact and allowed people to scale businesses and not have to set up server farms and all this type of stuff, it was largely the end people maybe didn’t feel that except they got more apps and businesses and things like that. But it’s the day-to-day workers that I think are most impacted by today’s AI shift, and we certainly see that in the translation world.

Hannah Clark: Yeah, I can imagine it must be really amplified in that specific space.

So when we talk about implementing AI and translation services, like you mentioned, there’s a big impact on the workforce of translators, end users. How do you balance the efficiency gains with the impact that this has on your workforce and how has their role changed?

Andrew Saxe: Yeah, so when we look at as long as both, kinda a software, we’re both a SaaS software company and you provide tools to manage translations and the whole process. And if you can imagine those thousands of people or hundreds of thousands of people typing in translations, provide tools to manage that.

And then there’s of course, all the people who are actually doing the translations. And we offer those translation services. And largely people have been typing in those translations, but, or reviewing translations for quality and selling mistakes and all these types of things. Brand voice for an organization.

And a few years ago, like 10 years ago, like neural machine translation came out, which is a pretty big step, maybe five or six years ago, I can’t remember the timeframe. But Google Translate has been around a long time and they kinda switched to neural machine translation, which made, had a pretty big impact on quality, but it wasn’t the impact that allowed you to remove humans from that process.

So if you think about the process, it’s someone types in the translation, someone reviews it for quality, and then it goes back to the content owner. It’s always been that way, and even with machine translation was still that way. So with AI, we’re really looking at taking out the humans in that workflow, in that process, we’re at least changing their role, so humans are, as the AI gets better and the tooling gets better around it, the humans come out of maybe typing in the translation and we’re able to regenerate the best first translation. We’re able to use AI and LLM is to assess the quality of it and then decide if it should go to a human or not.

So the human role in it more goes into a validation type of role. Say when we’ve assessed the quality needs to be checked out. The human comes in and they’re like, oh yeah, this is maybe not great. And maybe they find all those edge cases that the LLM isn’t able to check or find correctly, and then you can feed it back into the machine and hopefully it improves overall.

But like the role definitely shifts from being in the beginning and the middle of the workflow to being at the end to do random validation or sampling of content for quality. So it’s definitely a pretty big shift in how people are working in. Who knows, like eventually it might get to a point where you don’t even need some of the validation components, but certainly that’s the mood that is happening right now.

Hannah Clark: Yeah, I find that so interesting, especially when you bring up the something like brand voice, like something that’s so nuanced that can be really difficult to pick up on and really master, even if it’s in your own language. And then to find that the, with the, the translation might be, I can see that being a lot more complex for something like AI to be able to adequately gauge and be able to translate.

So I can see how that layer is still very much relevant and it’d be very difficult to replace.

Andrew Saxe: Yeah, it definitely is. Definitely is. And yeah, the brand voice is like a huge component or special terminology. Like people have brand words and glossary terms they want to use, or words they don’t wanna stay away from.

And you sure that the translations are respecting all of that. And yeah, AI does a pretty good job of it. They can also have some missteps. You have to build kind of the guardrails and sometimes the human aspect of that is the guardrail.

Hannah Clark: Oh yeah, absolutely. Even just, like having used AI for content ideation and that kind of thing. There’s a whole lot. Yeah. I think that it’s interesting to think about what the technology could be capable of now, but yeah, it is interesting to think about like the, there were different roles of interference. That it’s not just a cut and dry, direct translation gig a lot of the time.

So as far as how it’s transforming from a human typing task, if we’re getting a little bit further into this validation role, so what does the workflow look like now versus how it looked, five years ago before we saw this whole age of AI transformation?

Andrew Saxe: Yeah, honestly it was like, and like we’ve seen even from the Smartling landscape, like translations used technology for decades and desktop tools and server tools and all these types of things to manage it. But there’s always been the human typing in and reviewing and assessing quality and things like that.

And it really is. And when we think about the cost that goes into translation and organizations spend hundreds of millions and billions of dollars on translation, and it can take time to turn around, translations of humans are doing it. So the workflow really changes in that it’s instant cost goes down like rapidly if you’re paying people and you’re not no longer paying those people.

Like the workflow is incredibly more efficient and it really is just dealing with all of the edge cases and things like that. And there are cases still, and even though the AI and things are improving, sometimes if you have legal documents, you may still want humans to look at all of it or all information that really needs to be like pretty exact.

So there’s still some roles for certain types of content that need humans, but I would expect that would change over time over the next several years.

Hannah Clark: Yeah. I guess as the trust in the technology grows and it gets to be more effective. Yeah, it’s interesting, like when we talk about ethics, that’s a space where there is a little bit of a internal debate where, you know, like ethically where do we stand on getting to AGI and those kinds of advancements, so what ethical con considerations have come up in your leadership discussions around AI implementation internally or within the translation service industry at large?

Andrew Saxe: Yeah, so I would say that like for us, when we think about like the, at least the linguists that we work with and things all the time and we’ve been able to, I think, pretty successfully shift people into different roles and make sure they’re still making the same amount of money and things that they were before. Just maybe they’re able to do review more content and because it’s, requires less editing and less review or things like that, so like their velocity has improved.

At just a different end of the workflow. So like it’s something that we’re, and we’re at the beginning of really using these tools. That’s something we’re gonna have to really consider a lot more. And there’s certainly in the industry thinking about like, where are all of these people going to end up?

And like translation is a, it’s a very nuanced business and as you were saying, like just ’cause you speak Spanish or maybe you speak French, something like it doesn’t mean you can translate documents and specialty content and things like that. It just may not come out right. So it is a very like kind of a thoughtful exercise.

So I think there’s still space for that and it may even go more into creating content in language rather than editing existing content for translation. So there’ll certainly be roles for people still in the translation space.

Hannah Clark: This echoes in many ways, that content space as well, where there’s, now that the tools exist to make the process more efficient, the skillset that’s in demand and the space has also shifted where, it used to be that output and attentive to some of the technical requirements, especially SEO and that kind of thing. Were very high in demand. And now I imagine it’s similar in the translation space.

It sounds like the ability to be perceptive to specific nuances or linguistic nuances or style or being, very detail oriented around what a client, their requirements are, rather than just being able to just translate, word for word. It’s interesting how these kinds of technology shift have like a domino effect on the whole industry and kind of what is in demand.

Andrew Saxe: They definitely do, and even as we look at some of the research and things that we’re doing, and we’ll probably deploy this year or whatever, there’s a lot of trying to capture some of that nuance, like being able to say, how does your organization perceive itself or how do you want customers to talk about you?

And like coming up with some of those types of descriptions and things can really influence what the AI or LLM, or whatever is providing. And we’ve seen changes in quality based on that. So like as we’re looking at how humans are right now, even in some of the validation and review areas are still like involved in some of that nuance.

Like that will also shift probably over time.

Hannah Clark: The way that I like to think about it is, wherever the market kind of creates an influx in one area, so it could be volume of output. There’s going to be, a, sort of a pendulum swing into demand for higher quality, more, authentic insights or more kind of what’s gonna make this stand out versus a competitor piece when they can have the same tools as you.

Such an interesting time to be alive.

Andrew Saxe: Yeah. Yeah. It’s super interesting and it’s like seeing the dramatic shift in such a short amount of time is good.

Hannah Clark: Yeah. Yeah, it’s, it really feels sometimes as if we’re witnessing a shift that would normally be observed over multiple decades, and it’s happened over the course of a couple of months. My head is still spinning, and it’s so interesting, every time I have conversations like this, I feel like we have an AI conversation on this show, but once a month and every time it’s crazy.

Let’s get back to workflows because I’m curious, when you’re making decisions around deployment and how you change your workflow. As you said, you wanna really enable velocity and I think that it’s really cool ’cause it sounds to me like a lot of folks on this workforce are really now being empowered to do probably some of the more rewarding work rather than some of the more, the menial tasks that can be outsourced.

So how do you decide, what workflows or what tasks should be primarily human driven and which should be relegated to, this is an AI task and there’s an expectation that you’re using AI for this workflow.

Andrew Saxe: It really depends. And like in there is this, in the translation world, there’s this cost quality, speed matrix that people always try to navigate to get the best quality at the lowest price and fastest as possible.

We really do a lot of research and studying and testing to make sure that we’re offering the right tool for the right type of content. So we really have kind of these confidence levels that we look at, and especially in our AI tools, where we’re confident that a certain set of technology or tools or a workflow process will produce a set of content that we can guarantee that quality around.

So if we can guarantee the quality around a certain set of process or a certain workflow or process, then we like use that with the customer to pick the workflow that they should use. And some customers really do have a requirement that went, no, I want a human and a, I want two humans involved in this and maybe they have certain types of content that require that.

But really like we’re seeing just being able to more broadly apply technology enabled workflows to really all types of content. And with one caveat, sometimes there are like long tail languages that maybe don’t have maybe enough information in the LLM or whatever it might be to like really have an understanding of what the translation should be.

So maybe there’s some long tail languages that still require humans, but really like we’ve seen like being able to broadly apply these tools to really any type of content and still be able to guarantee the quality.

Hannah Clark: Yeah. I’d like to zoom out a little bit also in the organization ’cause we’re focusing on a specific, like a specific team really. When we’re zoom out at the organization at Smartling at large, how are you guys approaching adoption?

Are you finding that because I mean at this point everybody has a use case for AI. Are you finding that the organization is adopting AI in a fairly uniform way? Or is there some differences in terms of where it’s more applicable or where there’s more pushback on, doing things the old fashioned way?

Andrew Saxe: I guess like maybe two sides to that one, as a software provider with customers, or customers are like, they often have mandates internally to ask for AI. They want to, they’re not even sure how they might use it, but they have organizational mandates to make sure that they’re getting efficiency and their vendors and whoever they’re using are using AI and being able to report on that efficiency.

So like customers definitely want that and they’re asking for it, and we’re trying to figure out what those solutions should be that get gets some of the efficiency or cost savings or whatever they’re requiring for their organization. And then just like even internally, just like how we use AI and AI tools, like I know every team is using it, is using AI tools and certainly our, and our engineering team is, and our product team is.

Maybe to a lesser extent, but certainly still using AI and I know that our customer support tools are all AI-enabled and our help desk is AI-enabled. It’s like really everything is moving in that direction.

Hannah Clark: I’m seeing more and more of that, the AI mandates and I think it to right now, it’s interesting, I’ve had a few conversations about this so far where there’s a few different approaches that organizations are using, and I’ve heard it described as a carrot or stick approach. Where it’s like you can either incentivize AI and if that doesn’t work and the adoption rate is still not quite where you want it to be, then it becomes a mandate.

And that, I think that the, the purpose is to try and get people to be explorative and use, try and use the tools and get comfortable with how they can support their workflows. But I could see that also having a bit of a.

Andrew Saxe: Yeah. Yeah. And internally, we definitely put in some policies and allowed tools and usually our, whatever vendors we’re using have already gone through security audits and all those type of things so that we can have some safety and things around that.

But there’s a new tool every day that everyone could use and I’m sure trying it out.

Hannah Clark: Yeah. So if we talk about the hype cycle, so this is, the new technology, there are mandates, there’s excitement around it, there’s a lot of hype. And I think right now we’re coming off of a, I’d say a, the great skepticism is what I guess I’ll call it, of, what’s hype, what tools actually need AI features.

And this is, this is something that happens every time there’s a giant advancement. We saw it with social media, Web3. So how are you helping the organization kind of separate hype, trying to ground it and where’s the real value in these tools and trying to, ’cause I think there is the other side of the coin where there’s folks who are reluctant to adopt and then there might be folks who are maybe a little too eager kind of.

So how do you reign that into, what’s like, where’s the use case for AI that is valuable and not detrimental to quality, or?

Andrew Saxe: One of our AI researchers and engineers he’s always if you can, not everything has to be AI. If you can do it with a rule or if then just don’t spend so much time thinking that everything has to be AI-enabled.

I think like when we, and it is like when you look at, when you use tools like ChatGPT or something and you’re just amazed that like it just wrote this huge paper for you or you typed in something that came up with all this information. That’s one aspect that like creates some of the frenzy because it looks like it’s just doing all this amazing stuff.

But when you actually start to apply AI to like business solutions and things for customers, like you need to have a different level of confidence in what it is seeing. I think that’s where you’re like, when you start to see maybe the skepticism a little bit and you’re like, oh, this isn’t just like typing something into ChatGPT and it getting a great answer out of it.

It’s like figuring out how to apply the technology to something that improves your business or across that your customer’s outcome. So I think there’s certainly the frenzy and certainly everyone wants it, but there is, once you get into building some of the tools, there’s a bit more measured response.

You want to make sure that you’re building something that is actually beneficial for people and that will be useful. And sometimes that takes more time than just typing something in ChatGPT.

Hannah Clark: Yeah, absolutely. I feel like there’s a parallel here with social media and when that kind of came out in mass where there’s, this social norm.

Every business needs to be on social media and some people were not taking it seriously at all or not enough. Others went the other way where it was like you’re over-indexing in your social media. Where to the point where it’s like, where’s the ROI on? So yeah, I think there’s that fine tuning the balance, having a playground and encouragement to explore and get people to zero in on where the good use cases are.

But then, and to only hold onto the ones that are, they really have a proven benefit to the business.

Andrew Saxe: Yeah. And when we we build, we do lots of experiments and things and like we’ve been testing one that we think will be really great for customers. We always try to get like an 80%, 85% confidence for a few months.

It’s been like a coin toss of whether it works or not. And AI is great, but it’s still like 50/50 for other, produces the result that we want. So yeah there’s definitely like that trust layer and confidence of, and the right use cases for the dev technologies.

Hannah Clark: So looking at industries outside of translation, if you’ve care to comment, what parallels do you see in how AI is being integrated?

Just in the product space in general, are there lessons that you have from your own experience in your career that you think are more broadly applicable, that are informing how you’re seeing, as we move forward with this technology?

Andrew Saxe: A little bit. I think like I try to keep up with the latest tools for product and product managers and things.

I think that there’s a lot of things that I haven’t seen yet where it’s been like super helpful in maybe some the product development. And I don’t say that because I think like at least where like we’re supposedly inventing things that maybe don’t exist yet and new tools and interfaces and things that hopefully people haven’t seen, or new technologies or whatever applications.

I think it can be useful for like writing. Give me the outline of a spec and it’s gonna be this thing. Or it would be great if it could turn that into Jira tickets or something like that. So remove some of that complication of the, of what PMs end up having to do. I think there’s like some cool tools that I’ve seen for like just drawing something on a piece of paper and it turns it into a nice mockup and can point out different areas that you can focus on.

And so I think there’s a lot of application that we will, we’ll start to see that’ll improve. The kind of just some of the, I don’t know, stay to day in the weeds type of things that maybe require, maybe don’t require as much thought as like inventing something new.

Hannah Clark: Yeah. And like to the point of inventing something new, something that I’m very excited about is this five coding trend.

There’s so much potential that I feel is being unlocked now for collaboration within teams to be able to, have an idea and be able to generate a prototype that can be iterated on by a team that really has a skillset to take it to the next level.

Andrew Saxe: Have you seen some of those tools in progress?

Hannah Clark: Yeah, our internal team has been playing with it a lot and it’s really cool to see what people are inventing. But I think what is really neat is now I feel like we’ve all kind of had that moment as smartphone users or whatever, where you’re like, ah, there should be an app to do this thing that I specifically want.

Probably no one else. And now it’s you have the technology to try it out and see if it’s, oh, this is actually a stupid idea. Or maybe there’s, I have one. I can’t reveal what it is.

Andrew Saxe: It’s a great, yeah, that’s a great way for test to see if your idea actually does pan out or not.

Hannah Clark: Yeah, exactly. Like I think it’s cool to be able to instantly validate something that you know it, because that’s where ideas come from and often it’s someone’s probably doing it, but maybe you could be doing it better.

Andrew Saxe: Exactly.

Hannah Clark: As far as the relationship between AI and human workers, so as a leader, we’re talking about some of the impacts that you’re seeing on the organization right now. And I think we can both see where that’s going for end users and for folks at large. But kinda what would you like to see? What excites you about how this is enabling your workforce and what would you like to see in the next three to five years?

Andrew Saxe: I think from like a platform perspective, there’s, at least in the translation world and a lot of things, there’s like a lot of just like management of things. A lot of people are like in the weeds all the time, or a lot of platforms people spend their entire day in, and that is is their job.

And I think it’s turning into more of a managed by exception and that if you can trust that the AI or LLMs or whatever you’re using is creating the outcome that you want, then really you should just be managing open when it doesn’t create that maybe didn’t create the outcome that you wanted. So repositioning and reimagining a lot of tools, including platforms like Smartling, to be more of a managed by exception type of thing, rather than a, and also like more places. If you don’t need people to do all of these things, then like the technology and the tools and all these things can be in more places than just maybe inside the probably window or whatever.

Hannah Clark: What do you think of this hot take? So I’m, because one of the things that I’ve long been critical of just in general, is I feel like a lot of the time, the tradition, when it comes to your career trajectory is that people who are really good at a specific task within their, individual contributors tend to rise through the ranks based on their skillset, within their core function, and then become people leaders and lack the people leadership skillset.

I feel like this is the first time that, because we are now in a position where ICs are overseeing technology that are executing base level tasks, we are actually starting to train management at a IC level. This is like a skillset shift that people are starting to develop an actual ability to manage or an understanding of what it takes to oversee a team before they reach the people management layer.

Andrew Saxe: That makes total sense. Actually I haven’t thought about that, but like it’s something that I will certainly think about and even looking at the Smartling team.

Hannah Clark: Yeah, I think it’s interesting ’cause it’s, the nuances of managing human being issues, but I think it’s cool to see how people are starting to think like managers before they become managers.

So I’m excited for that shift as well.

Andrew Saxe: Totally. That makes total sense. Yeah. Yeah. I does create like a lot of new, maybe that’s just that it creates a lot of new opportunities for humans to do with things that they wouldn’t have done before.

Hannah Clark: Yeah. So interesting to see these unexpected sort of outcomes that happen when a technology has completely created a paradigm shift.

Andrew Saxe: Yeah. Sure.

Hannah Clark: This has been really fun. I really appreciate you coming on the show. I love an animated AI conversation.

Andrew Saxe: I’m sure you’ll have many more.

Hannah Clark: Yeah, probably next month too. Where can people follow you online, Andrew?

Andrew Saxe: They can find me on LinkedIn. I think it’s just ASaxe.

Hannah Clark: Wonderful. Thank you for coming and appreciate the time.

Andrew Saxe: Thanks so much.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to the Product Manager wherever you get your podcasts.