How American Express 4x Its Experimentation Velocity in 1 Year

Hannah Clark: Every product leader faces the same impossible choice—move fast and break things or build sustainably and risk getting left behind. But there’s a third path that most teams never consider. What if you could move faster by being more systematic? What if the secret to outpacing your competition wasn’t about cutting corners, but about building an experimentation engine so efficient that you could test and learn at a pace that makes your current sprint velocity look like you’re standing still? And what if you could learn how to do it from a product leader at one of the most recognizable companies in the world?

My guest today is Jean Castanon, Vice President of Digital Product at American Express. Jean has spent over a decade at Amex, working across multiple functions before focusing on digital product strategy. You’re about to hear how his team quadrupled their experimentation velocity in a single year, a counterintuitive lesson about what makes experiments “successful”, and how they built a referral program that went from being nothing to their second largest customer acquisition channel globally. Let’s jump in.

Oh, by the way. We hold conversations like this every week. So if this sounds interesting to you, why not subscribe? Okay, now let’s jump in.

Welcome back to the Product Manager podcast.

Jean, it’s such an honor. Thank you so much for making time for us today.

Jean Castanon: Thank you for having me.

Hannah Clark: Can you tell us a little bit about your background and how you ended up at your role, where you are today in Amex?

Jean Castanon: Sure. So I joined American Express back in 2011 as an analyst in our strategic planning group in London.

Over the years, I had the chance to work across many functions within the company, all the way from marketing, business development, and multiple digital functions. So I’ve worked in digital now for about 10 years in different capacities. Both, I would say more delivery focused, some more strategic. And what has really driven me over the years, I think, is the ability to make a tangible and meaningful impact for both our customers and our shareholders, and driving what I call purposeful innovation, which hopefully drives marketplace differentiation for the company.

Hannah Clark: Cool. Well, today we’re gonna be looking a little bit into the future and focusing on what’s going on with new technology and products and trying to get ahead of what’s currently competitive.

So to get us started, can you tell us a little bit about what forward thinking with product strategy looks like specifically like at an established company like American Express?

Jean Castanon: So we think maintaining a forward thinking approach is really important to ensure we’re meeting our customer current needs, but also accurately dodging what they’ll need in the future.

So one of the best ways that I think has worked and I would recommend other product leaders to have, is to develop product strategies with forward thinking approaches that are leveraging constant experimentation. And why that is important is that you can see what customers are responding to, and you remove subjectivity in that process by measuring their actual behavior.

So as a company, you know, without the traditional bank branches that others might have, our digital presence is our storefront. So we have to make sure that our digital experiences live up to our brand vision of providing the world’s best customer experience every day. So to do this, what we’re doing right now is we’re really turbocharging and expanding our experimentation efforts and we wanna build an always on ecosystem that’s constantly optimizing itself based on user behavior.

And this is especially important. They’ve been, you know how some of the younger generations like Gen Zs and millennials are interacting with us digitally, and they already make up over 60% of our. New customer, uh, acquisitions, but also they have higher expectations of what those digital experiences need to be.

So we need to constantly be uping our game to meet those needs.

Hannah Clark: I imagine that you’re doing any TikTok answers lately?

Jean Castanon: Not yet.

Hannah Clark: Not yet. That might be an over thing anyway. I’m not sure people are still even doing that. Okay well, let’s dive into the experimentation journey. So you’re scaling from 30 tests to over 120 tests this year, which is crazy. That’s massive growth. What drove this decision to ramp up the testing and experimentation and how do you build the organizational muscle to support that kind of growth?

Jean Castanon: You know, scaling that much in just one year in the US alone is a big leap and it’s being driven by, by a number of factors, but predominantly two.

First is the increase that we’re seeing to our site in terms of traffic, which basically gives us a much larger service area to optimize the experiences for. Second is our collective belief that experimentation is essential to keeping pace with evolving customer needs. So what this scaling looks like in practice, you know, is we had to structure teams, workflows, and tools to make testing a core capability, not just an afterthought or something that we do from time to time.

So we reorganized around a dedicated experimentation team whose hopeful focus is to identify, build, and run high quality tasks. And they are also backed by a strong cross-functional support structure that is aligned around a single North Star metric, which in this case is a 10% lift in conversion rate.

So that focus combined with the support from leadership and the sponsorship as well as the investment that it requires. Has been really, you know, behind what’s helping us to turbocharge that velocity, but also the impact of the experimentation program.

Hannah Clark: Hmm. Okay. So I’m really interested in the nuts and bolts behind aligning a whole team in order to meet this 10% conversion lift.

How do you align these multiple functions? We have product, we’ve got design, technology, analytics, marketing, and get everybody sort of on the same page working towards this one specific trajectory. How does that work?

Jean Castanon: It’s not always that easy. I’m not gonna lie. But I do think what is important is to start by defining a metric that will resonate with all of the parties.

So in this case, what we did is when we looked at, you know, from a business perspective and marketing conversion is a measure of growth. So that it was a, a meaningful metric that resonated with those functions for product and and design. It was a metric that resonated as well with our typical objectives.

Because it’s about eliminating friction in the user experience and ensuring that the product works hard and it’s meant to be. And then finally, from a tech and analytics perspective, conversion is a great metric because it reflects how well the systems, you know, availability and performance metrics are working as well as the insights are performing.

So conversion became like this common language for progress and one metric that united the different perspectives to make sure that this ship kind of sailed in the same direction. So we’re also focusing in addition, you know, to having that single North Star metric on creating transparency across the team.

So we share insights, tracking performance, we establish regular ceremonies and we identify what’s working and what’s not, because in itself that’s always evolving as well. So that clarity, I think, really allowed us to operate as one system rather than a set of disconnected teams with different competing goals.

And the result is a much more coordinated approach to optimizing the customer acquisition funnel.

Hannah Clark: I would say that this is the dream. Like I think that this is a very, very common issue, even with smaller scaling organizations, to be able to kind of work in lockstep across departments without this, you know, siloing getting in the way of people’s overdoing work or under, you know, not completely aligning on those functions.

So I think that’s really innovative. So let’s talk a little bit about team capacity. ’cause it’s, it’s one thing to align all these folks that kind of towards this one North star metric, but then managing the workload and ensuring that folks are working sustainably at a measured pace and not getting, you know, pulled in five different directions.

That’s a kind of a different task. How do you maintain that kind of focus in a larger organization when there’s always these competing priorities?

Jean Castanon: So I would say the, the most important thing that we did here is that we established a dedicated team that was solely focused on experimentation. So every day when they wake up, they think about this and it’s pretty, their core focus.

So this ensure that they aren’t pulling all of the other projects that keep popping up as priorities evolve. And their entire remit is just to run, optimize, and scale experimentation efforts. And the second thing that they do is that they also facilitate core teams to run experiments on their own by providing subject matter expertise and project management support.

On top of that, what we’re doing is that we’re really looking at this from three angles. The first one is from a talent perspective. So we brought together the right cross-functional mix, not just across product, but across design, analytics, engineering, and many more functions including legal. But we’re also upskilling team members to say ahead of how the external landscape is evolving.

Second is the process. So we’re deeply examining each step of the full lifecycle. Of running an experiment all the way from having an idea through discovery to build execution and all the feedback loops associated with this, and identifying which are the steps that have the bottlenecks and fixing them so that they don’t slow us down.

And the last piece, I would say just the third one, which is very important, is tooling and technology. So we’re obviously constantly assessing what our tech stack is here, but we’re also partnering with external vendors that specialize in experimentation. Platform so that we can move faster and stay focused on what we do best.

Hannah Clark: I see. Okay. I wanted to press a little bit on the results portion because I think this is something that every organization, whether they’re a startup, whether they’re scaling, whether an enterprise, we all kind of deal with this pressure to move quickly and especially in this climate where, you know, innovation is moving faster than it ever has.

The cycles of development are just at light speed at this point. And so there’s an enormous amount of pressure, uh, to change courses when experiments don’t immediately show lifts or results in the short term, even if we’re quite confident that progressing with that experiment and kind of giving it some time to mature could yield long-term results.

So how have you changed the mindset organizationally at American Express, and what advice would you give to other leaders who are kind of facing some of that similar sort of cultural and economic pressure to kind of yield results from experiments and keep moving?

Jean Castanon: So I think that there’s two pieces there that I’d like to touch on.

The first one is on the speed. I think speed is really important because the faster you learn, the better so that you don’t actually spend nine months learning if it’s actually not gonna drive the outcome that you want. But then the second one, which I think is often what is the biggest challenge, is this perception that if an experiment doesn’t drive the conversion lift, it’s been a failure.

And that’s not the case, right? And that takes a little bit of time. And we’ve shifted that culture by reframing what success looks like and we position learning as a valuable outcome in itself, not just a conversion lift, which is our, still our North Star metric. So I mentioned earlier that we have that 10% lift in conversion.

We also have a robust scorecard that looks at other experimentation outcomes, which includes velocity, win rate, the number of variants that we’re testing, et cetera. So my advice for other product leaders out there is. It’s ready to invest in building a learning culture early. It takes time and it’s something harder to test.

So often the default is to launch and just learn on the back of it. So make space for the teams to test ideas without fear of failure, and ensure that you have strong alignment on both purpose and process.

Hannah Clark: All right. Well, that sounds, uh, very sound as far as, uh, a process. I do wanna talk about something that maybe a little bit more fun.

We’ll go into a case study for the American Express Referral Program, which I know has grown to become now your second largest customer acquisition channel globally, which is huge, especially for a company the size of American Express, but it scaled from almost nothing over 10 years, so enormous success there with that program.

So what were the key product decisions that made this program successful in your view, and how did you scale it internationally?

Jean Castanon: Believe it or not, it actually started in international and it started in France and it then scaled globally. But one of the key product decisions we made about our referral program was to manage it as a true product from the early days, meaning that we thought deeply about the end-to-end experience and optimizing for the user journey.

And to do this, we focused on a few core things. You know, like the first, I would say the core infrastructure was really key and we needed to make sure that it’s flexible and scalable underpinning everything that we do. And then the second one is really about simplification. So making it easy for the user and making it very intuitive and seamless.

For example, right now what we’re doing is we’re testing tactics like QR codes for in-person referrals as well as asynchronous referrals where you might want to refer somebody but have them check it out when they get home. So the key is really about facilitating the virality that we get from the program while maintaining the right behaviors.

The next one I would say is personalization, uh, which has been obviously a hot topic for many, many years, but. We use many tactics here, including AI and data analytics to personalize the incentives that we give to our members and their friends, but also to personalize the experience that they see. And the last really underpinning all of this is controls.

So the same way that we have reality that is good behavior, we can have the opposite behavior. And facilitating that virality requires us to have the right control so that the program attracts the best customers and, and who we want as a result of this. I would say multiple pronged approached over the years.

We continue to see a lot of success with the program, especially with the younger cohorts of car members where we see about 75% of our referrals are resulting in millennial and and Gen Z acquisitions. So this aligns with what we’re seeing in terms of also digital behaviors. These cohorts are very engaged digitally and they’re also themselves used to referring more, which is fueling this, this viral loop.

Hannah Clark: Okay. I wanna talk a little bit more, ’cause you mentioned that AI is kind of one of the kind of tools in your toolkit right now as you’re moving through these processes. I’m curious, what kinds of use cases have you been piloting? Can you share what you’ve been exploring and how you’re kind of evaluating whether or not those, uh, the results that the AI use is, is yielding, are, are worth moving forward with or iterating on those?

Jean Castanon: I think from a, from a generative ai, it’s, it’s relatively early days, but it’s definitely gonna have very practical implications about our product development and experimentation processes overall. Right now we’re piloting use cases that are focused more on internal productivity. Things like speeding up, how we we’re streamlining ideas, writing them into testable hypotheses, designed to code translation, and then our ability to iterate faster.

So we’re able to experiment with these tools and with generative AI because we have thankfully, a technical enablement layer that provides us with the right guardrails to do that, which allows our technical teams to focus on innovation without reinventing controls. But I would say it’s, it’s still early days.

Hannah Clark: Well, I’d like to talk a little bit about your storefront as well. So you mentioned that your storefront is kind of your digital bank branch. You know, you don’t have physical locations for people to visit. How do you approach the personalization experience for prospects who are new to American Express?

Given that you rely on very different signals than you know, folks who do have more of a storefront situation.

Jean Castanon: You’re spot on that we don’t have physical bank branches like all our regional banks. So our website is effectively our digital bank. And for new prospects, personalization can become tricky as we would rely on digital signals typically to personalize those experiences.

So what we’ve done over the years is build a digital system that allows us to personalize dynamically even as the user progresses through the acquisition funnel. So then as we learn more through their interactions, behavior, declared inputs, we can adapt the products, the features, the offers they see, or even the experience.

So we’re bringing this test and learn mindset that I was talking about earlier to this as well, and, and our digital storefront is becoming a space where experimentation is really core to how we evolve the experiences, not just the tool that we, we use occasionally.

Hannah Clark: All right. And, and we always wanna talk about trends.

We said before we’d be talking about looking two steps ahead. So when we look at the future and the trends and technologies that you believe now could be most, most disruptive to the financial services customer acquisition space, how is American Express positioning itself to stay ahead of those changes?

Jean Castanon: There are a few major trends we’re watching closely, and this is not the full list, but the ones that come to mind. One is obviously the evolution of customer expectations around immediacy and, and relevance. And I think as AI will become more embedded in everyday consumer tools, that that this will become even more important.

So we need to continue prioritizing that personalization that I was just talking about, but also the ease of use of our experiences and of our core acquisition channels such as, you know, our referral program that, that we’ve talked a little bit about before the second trend. That is also something we’re watching very closely, is the way that users search for information online.

And this will continue to change and it’s been changing for the past few years. So we’re evolving our strategy to stay ahead of the shift. Uh, what search will remain a key way we engage with users, and it’s shaping both how we think about this from a technology perspective, but also from a content strategy so that we keep up with the changes of core consumer behaviors.

And a final trend I’ll mention is the change of competitive landscape. We’re not just fighting for share of wallet and attention with the traditional issuers. Fintechs are now prolific and they’re really defining also what user expectations are from a digital experience perspective is. So, I truly believe that our ability to test and learn at speed will become a differentiator.

So that’s why we’re investing so much in this infrastructure that we were talking about earlier in the podcast because modernizing our digital storefront and our branch experiences as well as upskilling our teams across all the functions that make this a reality to become more agile, is gonna be really, really key to win in the space.

So we’re very deliberate about future proofing this and why we build, so our systems and and processes are durable, flexible, and not reliant on one person. For one team. And I think that last piece is also very important.

Hannah Clark: Yeah, absolutely. And it’s an ongoing process to refine all of those, all of those aspects.

Well, thank you so much now, this has been wonderful. I really appreciate you taking the time out of your super busy schedule to chat with us. Where can folks continue the conversation with you online?

Jean Castanon: You can find me on LinkedIn. Thank you very much for having me today. I really enjoyed our chat.

Hannah Clark: Me too. Thanks so much. Take care.

Thanks for listening in. For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanager.com/subscribe. You can hear more conversations like this by subscribing to The Product Manager wherever you get your podcasts.