How Product Managers Can Wield Data Superpowers To Triumph Over The Competition

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Hannah Clark: In a perfect world, every decision we make would be data-driven. So what happens when the heart is willing, but the data we want just… isn’t there? Or the process of acquiring that data is so onerous, the juice just is not worth the squeeze? For a lot of product teams, too many decisions end up being based on one number only—the salary of the person making it.

Today I spoke with Vijay Umapathy, Director of Product Management at Heap. Vijay shares how his product team has become incrementally more data-driven, as well as the breakthroughs they’ve had to generate more insights in less time. He also touched on a really brilliant tactic his team has used to prevent riffs between product and engineering teams. Let’s jump in.

Vijay, thank you so much for joining us today.

Vijay Umapathy: Awesome. Thanks for having me. It’s great to be here.

Hannah Clark: Yeah. So we’ll kick it off a little bit about your background. Can you tell us a little bit about your ascent in your career and how you ended up at Heap?

Vijay Umapathy: Yeah, sure. So I started out studying engineering studied computer science at MIT, and I started out in product right out of school. So I joined the APM program at Google. One thing that’s kind of fun is I was one of those early people who was sold on the pitch of, “As a product manager, you’re the mini CEO”, which is completely not true and is like now viewed as this really archaic way of saying the field and it’s completely correct.

But I will say one thing, which is from the beginning, I really liked the idea of wearing lots of different hats. So like wearing a customer facing hat or like wearing a design hat, or an engineering hat or product hat, and kinda like swapping between all those different perspectives when it comes to helping people with technology.

And so that part has been true about being a product managers, wasn’t like, totally a false hitch of what it would be like.

Hannah Clark: So I wanted to ask you, there’s something on your bio, on your Medium account that says that you really love to create tools that give people superpowers. So I thought that was a really fun way of putting that, and I wanted to know how that looks in the context of Heap, which is, for those who don’t know, it’s a tool that product people can also use.

Vijay Umapathy: Yeah. Yeah. So Heap’s like a, it’s a product analytics tool. So just to give you an idea of what I think a lot of product managers experience with analytics is, I can actually tell you a little bit about what it was like at the last company I worked at. So I used to work at this robotics startup, and we had this like dashboard czar, there’s this data guy.

And so if I ever had a question about user behavior, right? Like how many people are using this feature, like what are they doing, where are they getting stuck? I would have to file a ticket for this guy and I’d have to wait for weeks for it to get prioritized. And this is by the way, just to create like a chart for me to answer this question.

And it was just like really archaic and it was extremely disempowering. And so, I think what really drew me to Heap when I was looking at it as a company to join is I saw, oh, okay, they’re building stuff for product managers. What’s special about it? And I think the thing that’s really cool with Heap is that they automate the work of manually writing code to track every user action that you care about.

So like for example, if I have an ecommerce store and I want to know how many people are clicking add to cart, usually with analytics tools, you have to write a line of code that says someone clicked the add to cart button, right? Then you have to maintain that over time. It’s like a whole mess. Whereas with Heap, we’re automatically capturing all of that data.

So we automate a lot of that tracking work. But the really powerful thing is if I’m asking a question six months later and I’m saying, ok, how many people did this action or used this feature? And I forgot to like bug anyone about about tracking this or make that ticket or to like the god of the dashboard, right?

Like I don’t have to wait, right? So I can ask that question. I have instant access to all the historical data and I can just answer the question, right, and so like in seconds. And to give you an idea of what’s cool about this is if you think about like how many meetings happen where someone says, oh, how many people are doing this?

And you don’t like, immediately have the answer, at your fingertips. And so then the loudest person takes over. So I was like literally in one of these meetings pretty early in my time at Heap where you know, we were like working on this page. It has a browse experience and it has a search experience.

Like you could think like a Google Drive, right? And someone was like, okay, well let’s assume a lot of people are just searching on this page. Strange assumption to be making. I said, all right, let’s talk about the future of search and redesigning this thing. So while this person starts rambling I go into Heap and I’m able to go make an event for people searching on this page.

I have instant access to all the historical data. Then I can make a chart that says, okay, what percentage of people are doing this? And I actually thought, oh, it’s only 27%. And I was like excuse me? It looks like the minority of people are searching here. Can we talk a little bit more about browse and it shifted the course of the meeting, right?

So that’s what I mean by superpowers. I think that’s I think we’re gonna look back 10 years from now and it’s gonna say, it’s like it’s absurd to like in the moment like that, not be able to answer that question.

Hannah Clark: So you personally have quite a passion for using data analytics and in decision making. How do you think some of your passions have shaped sort of the development of the product and some of those insights that you’re able to gain?

Vijay Umapathy: Well, I don’t know about necessarily shaping the product, right? I think a lot of times we’re leaning on our customers to do that, but I will say that it’s probably helped shape how we talk about best practices.

So I think one of the things that’s really cool is this ability to ask all of these ad hoc questions, one after the next. And what I’ve found is when you give people the power to ask these questions and to get answers really quickly, now the problem actually becomes your processes.

So a lot of people are set up to say, okay, when you launch a feature, you’ll sit down and you’ll define the five metrics that you’ll measure. You’ll then sit down after the fact and you’ll look at this dashboard, and you’ll then share this with executives in a meeting and you will move on with your lives, right?

Whereas instead, what we’ve tried to practice at Heap, we’ve actually made templates for this, is you should be really hypothesis-driven, right? So when you actually start out saying, I wanna solve a problem with this product or with this feature, we actually have a template for thing called a problem brief, where you basically say, I’m not gonna prescribe a solution.

I’m gonna describe the problem in detail. And at the end of that is what we believe is the most important part, which is a quantifiable hypothesis, right? Which is, I believe that this set of users is going to change their behavior in this measurable way. And maybe I guess if here’s how much I think it’ll change by, right?

Of what does good look like? And then we have this ritual that we created after feature launches that we practiced a bunch ourselves, and then we wrote templates and blog posts and everything about this called after action reports. And essentially what these are, you start with what was my hypothesis from before we built this and was it right?

Did the, did things move the way we thought they were gonna move? And then what else surprised us? And that what else surprised us, you can go down a rabbit hole. And I think that’s the part where I think Heap has encouraged that kind of behavior with our team, as well as with our customers, is going down this rabbit hole of additional questions.

What segments of users did very different behaviors, like what different paths did people take through your feature, through your workflow that you added to the product that you didn’t expect them to take.

Hannah Clark: Can you give us an example of a rabbit hole that you went down recently that sort of can give us a really good frame of reference for how that could, like, how the power that could really bring?

Vijay Umapathy: Yeah, sure. I think a really good one is when we, so we actually just launched a, like a session replay capability in Heap about six months ago. We acquired a company that does session replay, which by the way is like you can watch a visual reconstruction of what one user is doing, and so click by click, you can see what they’re doing.

And so when we launch that feature, we actually originally were looking at doing it as an integration. And so we built a bunch of entry points to it inside of our analytics features. And so we saw that a lot of those were getting, those were getting usage and people were going and trying it out. But we found was there was some friction when new accounts were setting it up because they were expecting workflows that were a little bit more qualitative first, right?

And so they were, we could see patterns where they were navigating to other parts of our site to try to get those. And so one of the things that we did as a result of that is we’re working on some iterations to make it a lot easier for them to navigate those kinds of workflows as well. And so I think this is actually a really common pattern in product is like you ship something, you have some set of assumptions and some things go well and some things you’re surprised by and it’s fine.

And then you keep iterating and you ultimately get to a place where your behavior normalizes in something close to the paths that you’ve paved, right? But usually when it starts out, it’s never like that.

Hannah Clark: Okay. So, switching gears a little bit. Something that has really been on my mind a lot is breakthroughs recently, like these moments in time where you have this aha moment that it sort of becomes the background noise of your processes. Can you tell me about a breakthrough that you’ve had with regards to using user insights and creating something really interesting with those?

Vijay Umapathy: Yeah. So actually I think the biggest breakthrough that we have achieved in our product that I think has also resulted in a lot of changes to how we operate as product teams and how our customers operate, it really comes from the session replay capability that we added to our product. So just to give you a little bit of context, right?

So usually what happens is, and we are actually an example of this, usually happens is you have your quantitative data and one tool, right? So you have like one silo that has all of your behavioral data, you’re making charts off of this, your funnel reports like all of this, right? And then separately you have this other silo that has qualitative data, right?

And there may be, by the way, two different teams that are looking at these two different silos of data, right? So maybe your design team is going in and regularly, watching some session replays, right? But what ends up happening is, you have this pile up, right? So you have millions and millions of hours of session replays, more than you could ever conceivably watch, right?

But one of the big problems is that you don’t know which sessions you actually should be watching. Right? And furthermore, you don’t even know, let’s say you open up one of these sessions and it’s an hour long, right? What moments in there are actually important, right? And what moments are in there are important in a way that actually impacts your business in aggregate, by the way, not just looking at one user but across several users.

So we actually used to have this meeting where we would sit down every Friday and we would watch, a few different session replays. This was a normal process. I know tons of product teams that are doing this today, right?

And we would do this kind of with the goal of building a ritual to have empathy with the customer, and then also at the same time maybe targeting the discussion around a specific workflow that we cared about or we’re working on in general to come up with some ideas, right? What are some potential things we could change that might be impactful, right?

So that’s what we were trying to do. Here’s the problem: there’s one poor soul who had to do prep for that meeting, okay? So that person, there’s a 30 minute meeting and with 20, 30 people in it, right? Expensive meeting. So one person has to sit there for three hours or so ahead of that meeting, and they have to go watch hours and hours of these sessions, most of which are useless.

And then handpick three that we’re gonna watch together as a group, right? Horrible, painstaking job. And by the way, it’s like really easy when things get busy to just, that person doesn’t have the time to do that, right? They’re not paid to do that. And so then the whole ritual falls apart. And now your whole usage of that qualitative data, like stagnates, right?

It becomes shelfware, right? And so when we acquired the session replay company oric, we did this like June of last year. When we integrated them into Heap, we didn’t just like slap on a session replay page into our product. We actually deeply integrated it into our analytics itself. So where any time you’re looking at any kind of quantitative trend, we essentially give you a show me button, right?

And that show me button will not only handpick the right sessions for the context that you’re looking at, maybe that context is, oh, okay, I just shipped a feature. Did it change between last week and this week? How did people click differently through it? Easy to answer, right? Or a bunch of people are dropping off at step three in my workflow.

What’s happening there, right? And so what we did was we deeply integrated it there. We gave you the show me button, and so then we handpicked the right sessions and we jump you to the right moment in those sessions. So here’s what changed with this meeting, right? So we brought back a version of this meeting, but as a brainstorm, our growth team actually led this brainstorm.

I still remember walking around dazed in my office after we ran this, because this team did no prep. They took existing funnel reports that they had for different parts of our setup and onboarding flow. They just took a chunk of different people and said, you go in this room, you go in that room and watch as many of these replays as you can and generate as many kind of like virtual post-it notes of areas of friction.

And so each team just picked a different funnel or a different step. And again, no prep work. So like that person with the three hours of sifting gone, right? And they’re sitting there, they came out with hundreds of ideas in 30 minutes. And so that was a game changer moment where we were like, oh my God, like this can help teams ideate so much faster than they could have ever conceived that they could ideate.

Hannah Clark: That’s fantastic. And such a massive impact, right?

Vijay Umapathy: Yeah, absolutely. And the cool thing is it’s data-driven too, right? I think that’s something that, that a lot of people forget is 90% of product decisions right now, I think in the world, are just made by people’s opinions, right?

It’s the, have you heard of the HIPPO? The highest paid first opinion, right? Like a funny PM acronym, but I think it’s very real, right? Like most of these things are just gut decisions. So even if you’re looking at some anecdotes and you created a post in one of these meetings, if that anecdote was informed by quantitative context, like a part of a workflow that’s not performing well, that’s a win.

Like you’re incrementally more informed than you would’ve been otherwise. And so that’s the thing I think people have to realize is that like you can win by taking a thousand small steps in the right direction. When it comes to being data-driven, it doesn’t have to be this big on/off switch. You’re either not data-driven or you are data-driven. You can slowly climb up that ladder.

Hannah Clark: Coming back to the idea of innovation, so how do you balance the need to innovate and iterate and all those kinds of things with making sure that the product that you’re delivering is always stable, reliable, and continues to keep that consistent customer experience?

Vijay Umapathy: Yeah. Okay. So I wanna riff on this question a little bit, if you’ll allow me.

Hannah Clark: Of course.

Vijay Umapathy: So for example, just to answer the first part, which is like reliability and stability. I think usually if you are monitoring like, basic kind of reliability and stability metrics, usually also companies will have a service level agreement with their customers about things like uptime.

In other cases, maybe performance, right? And if you’re monitoring these things, usually the data will tell you pretty clearly when you need to drop everything and go fix it. I think a much more interesting question that has a lot of subtleties that causes challenges for PMs is how do you navigate technical debt with your engineering team, against going and building net new product capabilities and solve problems for customers?

Like for example, right usually there’s a common like PM trope, right, where you’ll have a product manager who’s saying, let’s go keep building new features to solve all these problems for our customers. Executives love that person because all the ideas are really shiny and they see dollar signs, right?

And then the engineering team is like sitting there in a corner saying, we really need to upgrade our version of Node. And they’re just getting increasingly bitter over time as their idea gets deprioritized. I think this builds rifs between product engineering teams when they don’t need to be doing that. So one thing that we’ve come to practice on a number of teams at Heap is kinda using the roadmap to prioritize our technical investments in our foundations.

So we actually did this thing called Making an Agility Scorecard. So essentially we did, as we said let’s start with the operating principle that computers should work for us, right? Like it should become easier and easier for us to do the things that we need to do more and more often on our roadmap, right?

So step one is look at your roadmap and say, what are the kinds of changes that I want to make to my product more often versus less often, right? And then if I look at the things that I wanna do more often, every time I do that, am I sort of encumbered like that is every time I change something like this, I’m incurring tech debt.

Maybe it takes weeks, but it feels like it should take minutes and it’s super annoying. Or am I enabled, right? Like it’s relatively easy to do. It’s pretty straightforward, but it’s still work. Or is this whole thing optimized? It was designed for me to do this thousands of times a day, right? You actually don’t wanna be optimized for everything, because then you’re optimized for nothing.

So you want to step back and look and say, okay, what subset of this do we know we’re gonna do all the time? So to give you an example with Heap, one of the things we realized was we’re gonna be building new types of data visualizations forever, right?

And another thing that we realized was we’re gonna be building interfaces for people to ask you questions of data forever, right? And so we looked at those things and we said, all right, like, how long is this taking? How long should it take? What are some of the gripes that we’re seeing from our development team when it comes to building these things?

And we were actually able to, step back and say, actually we’re gonna pause some feature development on a couple of these teams for a little while and prioritize shifting our front end to react. Which is I know is a thing that a lot of companies out there have been doing over the last couple of years. These big painful projects. And by the way, it sucks. It takes a long time.

It’s like pretty brutal. There’s like lots of edge cases. But it’s important if you’re like, if you sit down up front and even with your executive team, you frame it as building velocity for yourself in the future, right? I’m investing a month now to take a thing that I know I’m gonna do a hundred times and make it go from one week to one hour.

That’s gonna pay off real quick. And so now you’re going from tech debt to like tech dividends, right? And so if you think about it like that, then you know, then it becomes this exciting collaborative thing that you’re doing between your product and your engineering team together. And now you’re not building this rift between teams.

Hannah Clark: So Vijay, what kind of trends are you seeing right now emerging in the product management and data analytics space? And how do you stay ahead of those trends to make sure that your team is always at the leading edge?

Vijay Umapathy: Yeah, so I think one of the big trends is breaking down silos between data.

I think that there’s a couple things that are driving this, right? So one is this idea of obviously broken workflows, right? So we actually talked about that earlier in that example of session replay. When you combine it with the context of quantitative data, you can get to value and you can get to insights dramatically faster than you could otherwise, right?

So I think that’s definitely like one factor is when you have the opportunity to make workflows better that are currently across multiple tools, like you’re starting to see more tool consolidation, I think the fascinating thing that’s accelerated that in recent months is all of the kind of the tech session, right? Like the kind of macroeconomic challenges for a lot of these tech companies that were assuming that they would just grow infinitely and the capital would be basically free forever.

So there’s this shift towards, okay, as budgets are tightening, CFOs and procurement teams at companies are really pushing on people to consolidate tools. And I think what’s important as especially for B2B companies, right? Like a lot, I know a lot of the product managers out there work at B2B SaaS companies, right?

And so you’re probably getting a lot of pressure to say, let’s go build some version of every tool in every space adjacent to our tool, right? Why? Because consolidation, right? But here’s the problem, right? When you do that, there is a huge risk of spreading yourself too thin. So you have to be really careful and thoughtful about how you do that.

So you have to step back and say, is there an opportunity for me to actually create a better customer experience by combining these tools? Am I just combining them for the sake of combining them? Or is there a real opportunity to create something better? And then more importantly, in addition to creating something better, is there something that I am uniquely suited to do to create something better?

That means that when you actually, when you go and you build something like this, and you go and build out a version one of an adjacent tool from an adjacent space that has existed for years. By definition, you’re gonna be behind, right? Like you’re not gonna have materialized like years and years of work overnight, right? But when you do that, you can actually, if you have a real differentiator and you have a real workflow that you’ve now improved dramatically for the end consumer, you have a hook to really start delivering value and to beat out your competition.

So I think it’s really important when people think about this trend of tool consolidation and silos being broken down. They’re thoughtful about how they solve those problems and how they build those features.

Hannah Clark: That’s fascinating. So just an out of left field since we’re about to wrap up on time, what’s on your playlist right now? What do you like to listen to when you’re at work or outside?

Vijay Umapathy: So, I listen to a lot of electronic music. I’m a big fan of Above and Beyond, if you’ve heard of them. They’ve been around forever, but they have a couple different record labels. So the one I listen to a lot that has playlists on Spotify is called Anjunabeats, and they have tons of emerging artists on there.

So there’s a lot of people who are just beginning to really blow up in their careers, and they’ve kinda created this whole ecosystem of great sounding music. So I usually listen to that. That’s what’s on my playlist.

Hannah Clark: Oh, cool. Well, I’ll have to check that out. So Vijay if people wanna keep up with you or follow your work, where can they find you online?

Vijay Umapathy: So you can find me, I’m on like Medium and you can find me on Twitter @vijayumapathy. And yeah, definitely make sure you check out Heap if you’re building digital products, we’ll help you make decisions really quickly. Hopefully talk through some examples here that help see how you can do that and be empowered and yeah, don’t let the hippo win.

Hannah Clark: Fair enough. Yeah. Thank you so much for joining us. We really appreciate your time.

Vijay Umapathy: All right, thank you.

Hannah Clark: Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The CPO Club wherever you get your podcasts.