OKRs Q&A Podcast Ep. 59: The Value of Critical Inputs and Outputs from Amazon’s Perspective | Steve Anderson
In this exciting episode of the OKRs Q&A Podcast, Tim Meinhardt brings back podcast guest Steve Anderson, author of The Bezos Letters. Tim and Steve discuss the difference between inputs versus outputs (and what Jeff Bezos’ opinion was on both,) his thoughts on Jeff Bezos’ math and judgment-based decisions, evaluating anecdotal data, why Amazon was so obsessed with data, and finally why they believe in empowering their employees to do great things.
Even John Doerr speaks about Amazon and their obsession with inputs and data in his interview with Sal Khan. To view this video, check out our resources page (www.atruity1.com/resources) or view it on YouTube at https://youtu.be/JYOyI8_r5sY
To order a copy of Steve Anderson’s book The Bezos Letters, click here:
If you are interested in working with the Atruity team or have a question you would like addressed on the show, please email contact@atruity1.com or visit our website at www.atruity1.com
Tim Meinhardt:
Welcome everyone to another exciting episode of the OKRs Q&A Podcast, also known as the OKR Corral, where OKR insight is the king. I’m your host, Tim Meinhardt, President and CEO of Atruity, an OKR consultancy headquartered in our nation’s capital.
Before we begin, if you’re an OKR fan and enjoy our podcast, please subscribe, leave a review, and explore our website at www.atruity1.com. And finally, should you have a burning question you’d like addressed in future episodes, please drop us a note at contact@atruity1.com.
In this exciting episode, I have the pleasure of speaking again with Steve Anderson, author of the book The Bezos Letters. It’s a terrific book and Steve’s insight is outstanding. And in this interview, we discuss in depth the difference between inputs versus outputs (and what Jeff Bezos’ opinion was on both,) his thoughts on Jeff Bezos’ math and judgment-based decisions, evaluating anecdotal data, why Amazon was so obsessed with data, and finally why they believe in empowering their employees to do great things. For OKR fans, this is a super podcast as Key Results are the inputs that drive Objectives, or the outputs. Even John Doerr speaks about Amazon and their obsession with inputs and data in his interview with Sal Khan, which is posted on our resources page and widely viewed on YouTube. So, everyone, please grab that cup of coffee, put in those earbuds, and enjoy my lovely discussion with Steve Anderson.
So, Steve, welcome to the program. This is your second time on the program, and I really wanted to have you back because I love your book, The Bezos Letters. And you know, it just resonates, I think in the OKR community. So, tell the audience a little bit about yourself again, if they don’t know who you are, and what kind of drove you to write this book.
Steve Anderson:
So, my background is the insurance industry, both selling, operations, and insurance agencies. For the last 25 years, I’ve focused on technology in the insurance business. And you know, what insurance agents need to sell better, connect with customers better, et cetera. And that work actually is what led me to this idea, that was the kind of the kernel of what has become the book, is that as I was looking at technology and how fast it develops and continues to develop; that is the biggest risk of business faces—actually not taking enough risk, meaning taking too long to evaluate a new technology, not jumping on quickly and those kinds of things. So that led me to start researching different companies, and we certainly know a lot of companies who are no longer around. Very successful at some point, and are no longer here, be that BlackBerry, Blockbuster, Sears, CompUSA, right? Lots of different companies. Kodak. Once successful, but not here. And so, what happened? And what about some of those companies that have continued to be successful, you know, through the last 25, 30 years, et cetera?
And that’s where I came across Amazon, certainly continuing to grow; interesting story of how Bezos started that company. But really, what caught my attention in my research was his letters to shareholders. His first one, it was 1997. They went public that year. This letter came out in the spring of 1998, and he’s written a letter every year he’s been CEO. Now, he stepped down from that position, actually, here in 2021. So, the, you know, 19 and 20 letters are his last letters as CEO. And what I discovered, you know, is that I think at his heart, Bezos is a teacher. I mean, he actually talked about what he was doing, what he was building, what I call principles; I’m not sure he would. I think he would call them the principles to growing Amazon, that I realized I think could apply to any company. So, I undertook this crazy project of writing about them, right? And that ended up being the book. And yeah, it’s garnered some success, and kind of still surprising to me. Wall Street Journal, USA Today bestseller. And I think maybe more interesting is it has been or is being translated into 18 different languages. In fact, our latest one last month, well, the Greek version was published in Greece.
Tim Meinhardt:
Well, fantastic. Wow, Steve, that is such a great story. And you know, you and I first met several months ago. You know, I had a chance to go through the book and I just thought it was so…what’s the word I want to say? I just thought it was so invigorating to read about what he thought and kind of how he went about things. And, you know, sometimes not focusing as much on all the financials, which sometimes people become financially obsessed in the business world. And, you know, one of the really interesting things—and I think it relates to Objectives and Key Results, and so I want to spend some time on this today—is this concept of input versus output. And you know, Bezos was really someone that was a methodical mathematician of measurement, you know, and so he believed that, and again, I’d love for you to expound on this, but he believed in measuring inputs, you know, and I look at inputs like Key Results and Objectives and Key Results. You know, the output is something that’s aspirational. We hope we get there. You set it up. It’s concrete. It’s long lasting, but it’s really those inputs of what you’re doing, and what was your opinion on Bezos and what do you think about him using the input method to be a measurement person?
Steve Anderson:
Yeah. Well, it’s very important to him, and he really did—you’re right—focus on the inputs, not the outputs. And he said it several times in interviews that new executives coming into Amazon from outside are surprised by how little they focus in meetings on the financial results, right, the PnL, whatever, you know, those are. And he says people are surprised because they are coming from an environment where that is the key, is the financial results, when he focuses on what are we doing down on the fulfillment center floor? What are we doing with all these other things that will lead to the financial results, but we need to focus on, are we executing well? And so, in my book, I lay out 14 principles and I grouped them into four different cycles: test, build, accelerate and scale. And Principal Number 13, you know, really addresses this; it’s measure what matters. Question what’s measured and trust your gut. And that kind of encapsulates this whole idea of…Amazon is massively data driven. I mean, they measure everything, but they also question what’s being measured.
And he talks about in one of his letters this anecdotal information. So, he actually published his email address in the shareholder letter. And people started sending him emails. Jeff@amazon.com, which, by the way, you can still do, you know, complaining about delivery or this didn’t come right or whatever it was. And originally, he looked at all those; he doesn’t now do that at all, but there’s a team that took that over. But he very much focuses that if there are a certain number of emails that are giving other information, even though their data is telling them, “No, this is working well,” they go back. And in fact, he was famous for it at one point in the growth of Amazon, where he would get one of these emails from a customer and he would forward it to the head of the product department; whatever, you know, was impacting that customer’s experience was just a question mark. And as a manager, you never wanted to get a question mark email from Jeff Bezos because basically, that meant you drop everything and figure out what happened with that one customer. And again, the inputs, not all our data saying, you know, “We’re at 98% on time delivery,” and all of these kinds of things, but what’s happening right here, and then engineering and going back to what caused that in the first place and correcting the source of the problem, not just fixing that one customer.
Tim Meinhardt:
Right, you know, what you’ve said, all these words resonate with the OKR community. And you know, first you talked about being able to execute, you know, how well are we executing? You know, we have a reflection period, you know, and we look at organizations and ask them, “How well are you actually executing the things that you’re executing based on the inputs?” The second piece that you mentioned in their principal number 13 was measure what matters. Gosh, it’s John Doerr’s book—it’s OKRs. And so, it raises that question that, you know, I think in the back of everybody’s mind is, “Are we measuring the right thing?” You know, and so we talk about it as an OKR journey because truthfully, you may discover as Jeff did, that, hey, when you start looking at things, are we merely measuring the right things, okay? Yes, we want to measure what matters. And so that chance to reflect a little bit, are we measuring it twice, you know? Or are we really, truly measuring the thing that really, truly matters? So, I just find this very fascinating. And, you know, John Doerr in his YouTube discussion with this gentleman from the Khan Academy, mentions Jeff Bezos and talks about his obsession with measuring inputs. And I just think that it correlates so well with Objectives and Key Results. You know, Jeff believed in math-based decisions, and judgment-based decisions. Can you just draw upon what you think compare and contrast these a little bit more so that folks can get a better understanding and why both are so important?
Steve Anderson:
Yeah, so right. Yeah, they both are, you know, and that’s why kind of the end of that principle is trust your gut. And trust your gut, right? So, data is not the end all, be all. But again, are we measuring the right things? And, you know, when you were talking, I was thinking about, you know, the dashboards, right, that are so popular now. Are the dashboards actually telling you what you think they are? Or are they sugarcoating or adding other stuff? And you know, are you really understanding what goes into, you know, whatever you’re looking at there? And then trust your gut. And that really goes back to decision making, I think, and what Bezos thinks about how decisions should be made. And again, that goes to another principle, which is in the accelerate, which is generate high velocity decisions. And so, in Bezos’ mind, there’s two types of decisions, and he calls them type one, type two, right? Pretty simple. Type one decisions are bet-the-farm huge decisions. And in Amazon’s case, their process for selecting another headquarters, I would say it falls into the type one type decision. And interestingly enough, Bezos says you should make those decisions slowly with as much data as you can gather and ultimately end up being a gut decision. But he said those decisions don’t happen very often. Said the most common decision is type two, and type two decisions are the everyday decisions, the quick decisions, the do we do this product or not, do we release it here or not? And he said the issue with type two decisions is you should make those with at most—and this is to me what’s really interesting—at most was 70% of the information you wish you had. And I think that’s a really interesting way to phrase it, right? Not 70% of the information, but 70% of what you want. And he says, if you wait too long, then you’re slowing growth.
Tim Meinhardt:
That’s fascinating.
Steve Anderson:
And isn’t that interesting? So, type two decisions are easily reversible. That’s kind of the principle. Type one are hard to reverse, type two are easily reversible, and they should be made, you know, quickly with high quality and small teams. And I flesh this out a little bit more in that he goes on to say high quality decisions and high velocity decisions. So high quality decisions are high standards—you hire the best, you keep teams small. So, what originally started as the phrase “two pizza teams.” So, at Amazon a team is only as big as what two large pizzas can feed.
Tim Meinhardt:
We use that right.
Steve Anderson:
They may have hundreds of teams, but that team is comprised of high-quality individuals, and they are given the authority to make decisions. Now again, for a large project like a Kindle or, you know, something like that, you literally could have hundreds of teams, but they’re small. So I think that’s interesting. And then again, the high velocity is…what he says is the problem as a company grows is that it’s really hard not to move into a type one decision process, meaning bureaucracy, hierarchy, you know, somebody lower on the organization chart has to go to a supervisor and then, you know, a regional manager, and then a VP and then…and all that does is slow decision-making down, not helping accelerate those decisions as you go forward. So again, that, you know, what you said was math based, absolutely. And judgment based from high quality individuals that are part of that team.
Tim Meinhardt:
Before we continue with the interview. We’d like to tell you a little bit about Atruity.
Voiceover:
At Atruity we understand the challenge of implementing a successful OKR program. While the methodology may be straightforward and easy to understand, the implementation and execution of the program can seem daunting. Your team is concerned because you’re unsure how to properly implement or manage your OKR program. You are not alone. This is where Atruity comes in. We know how to implement an OKR program and are experts in OKR implementation and management. By using our proven methods and implementation structure, we can help you to successfully implement OKRs within your organization in as little as 30 days. If your organization is considering implementing OKRs or struggling with the management of the program, do not hesitate to reach out to us at contact@atruity1.com. Remember, no plan succeeds on its own—execution is everything.
Tim Meinhardt:
There’s a couple of really good pieces in there, and actually I was interjecting that like, “Hey, we used the two-pizza rule!” And I tell people and I can eat a lot of pizza. But, you know, I think you come back to this interesting phrase that that we’re seeing more and more in today’s business world, is how to empower the employees. Look, the Great Resignation’s taking place. And how do you give people…they want to have impact in their day, they want to know how they are making an impact in that business so that when they go home, they feel good about themselves. And I think when you create those small teams, and you know, there’s a great book out called Team of Teams…is another great book. And that, to me, sounds a lot like what Amazon was all about—a series of teams, and a team of teams. And you know, I love this type of decision-making criteria because, you know, you do sometimes need to make quick decisions. And I think the thing with OKRs, and I relate it to OKRs, is things are written down, okay? They’re not just these amorphous discussions. And so, when everyone can see what everybody’s working on, okay, those conversations change. And that’s that visibility aspect of OKRs, which I get a lot of that out of Bezos, and a lot of that with them that there’s a lot of internal discussion—small teams talking about things. And it’s that coordination in those discussions, and I believe they probably had some elements of transparency in there as well, because that transparency begins to help with alignment and focus on really what’s most important and what we want to get accomplished. So, yeah.
Steve Anderson:
And there’s another phrase in that decision-making process that Bezos talks about and it’s disagree and commit. And so, he gives an example of the Prime Video team wanted to greenlight a new original show. And Bezos didn’t think it was going to work. And, you know, and so they pitched him the idea. He said, “I don’t think this is going to work, but I’ve hired you for your ability to know the industry and make decisions. I disagree with this decision, but I fully commit to support you if you move ahead with it.” And he said, “Think about how much time that saves. They didn’t have to go back and figure out how to convince me they were right, and I wasn’t. I just disagreed but committed to supporting them moving forward.” And again, and when you think about that in a normal business environment, right, today, that is not a mindset that often is…a company often adopts. It’s, you know, “I’m the manager, I’m the whoever. I know what’s best,” versus “I hired you because of your expertise. Okay, if you think it’s going to work, I hope I’m wrong. Let’s go forward.
Tim Meinhardt:
Yeah, I know in OKRs, what I find most fascinating is a lot of innovation comes from the ground up, you know, those good ideas come from the ground up and you want to build—and I don’t want to call it a safe space—but you want to build within your organization a place where it’s okay to take a risk and fail. Because some great ideas don’t necessarily work, but to squash it all and not give people that room to even say, “Look, I don’t agree, but I’m going to give you the latitude. Go for it. Let’s make me wrong.” That environment is so invigorating for an organization, and it’s what gets the best out of people, because people want that. They crave that, and being able to be successful because, you know, take the money aside, okay, and I go back to this resignation and why people are jumping ship and whatever. It’s that, you know, and managers today will say, “Gosh, I’ve lost grip. You know, my employees are remote. Heck, my clients now, and customers, they’re remote. They’ve lost grip. So how do I get,” you know, it’s the “Okay, let it go, let it go.” Make it in a place where you can understand and have things in writing that are transparent, where conversations can change, where you can empower your employees to be the best that they could be. And I get so much of this out of your book and your insight into what Jeff Bezos and the management team was looking at when they were, when they are, and are still building Amazon.
Steve Anderson:
Still building, yeah. And part of what’s interesting to me is, you know, Bezos has stepped aside as CEO to do other things. And I think one of the interesting things to see is, you know, their success when there’s succession, ultimately for the company, right, you know, is it just Bezos or has he built, you know, a strong foundation that now Andy Jassy can come in and put his mark on Amazon and help it continue to grow?
Tim Meinhardt:
Yeah. Well, they’ve certainly built a culture that I find absolutely fascinating. And, you know, like I said, John Doerr, who wrote, Measure What Matters? thought it was important enough to bring up the success at Amazon and the ability to measure input. And I want to go back there.
Steve Anderson:
Before you change subjects, I want to just interject this kind of the supporting idea of what you said in terms of failure. So, principle number one is encourage successful failure. And Bezos says Amazon is the best place in the world for an employee to fail because he understands that experimentation leads to invention, which allows innovation. And I’m absolutely convinced that employees aren’t afraid of failure. They’re afraid of the consequences of failure all too often. Meaning, right, that environment that you talked about, that understands that failure is part of the process of learning what needs to be invented on behalf of the customer.
Tim Meinhardt:
Right, yeah, so true, absolutely so true. You know what? I wrote down something else that I wanted to go back to real quick, when I was writing my notes and I was writing them so fast as you talk, I can hardly keep up with you, which was fantastic. And I’m a fast writer, but you know, we talk about this measurement issue. And you know, a little bit about quality and quantity. You know, when Jeff Bezos gets an email, you know, generally, I would think that’s about the quality of his delivery, hey someone’s upset, okay, so that importance of…but everybody always wants to measure numbers, you know, bigger, better, faster, less, more, blah, blah blah. But sometimes quality gets lost in all of that, and I think that quality piece is very important as well. What are your thoughts on that, Steve?
Steve Anderson:
Well, I think, you know, you’ve got to be careful, you don’t want just fast decisions, right? You want fast, high-quality decisions. And, you know, that to me is kind of the perfect way to make those decisions that actually help move the company forward. And I think it’s important, you know, again, Amazon is known as moving fast, and I mean, interestingly, being able to create market leadership in multiple different markets is pretty unusual for a company. But they also have a process in terms of how do we make those high-quality decisions? And it’s had various names over the years, six-page narrative, working backward documents. But when they’re making a decision, when they’re kind of coming to deciding “We’re going to build a Kindle or a new robot” that you know, goes around your home, they have a process that they go through, which is, the team proposing the idea is required to write down physically, a maximum of six pages document, that starts with a future press release and then includes frequently asked questions and some other components. And literally, that’s passed, printed out, and passed out at the meeting, not beforehand. Because what Bezos says is executives are busy. They’re going to say they read it, and they didn’t. So, they spend the first 15 minutes, maybe 30 minutes, of a meeting, everybody reading that document. So literally everyone’s on the same page and then they start asking questions. They do not—Bezos banned in 2004 any slide-oriented presentation. Keynote, PowerPoint, you will never see that at an Amazon meeting. And I think that’s really interesting because so many people think, “Well, that just slows things down.” But what Bezos says is, “Yes, it does, and it’s hard to do.” But when we do that work upfront, it’s so much easier, so you don’t get six months into a project and go, “Wow, we didn’t think about this or this or this” and have a failure at that point. It may slow you down in the beginning, but it speeds you up as you move toward the finish of that particular project.
Tim Meinhardt:
Yeah, interesting, you know, in OKRs, one of the things that we talk about is when you set your quarter, what is it that you want to accomplish during a given quarter? So, it’s that thinking forward, where do you want to end up? You know, that drives the ability then to move towards that, you know, everybody’s in meetings, it seems anymore to give reports. Rather than talking about what we’re going to do, it just focuses on what we did.
Steve Anderson:
And I would say, you know, kind of adding on to that, is what do you want to accomplish this quarter and what do you already know are the potential roadblocks, and what are the solutions you think might work? Or maybe you don’t have a solution to it yet, and that’s what you’re asking input for. Here’s something we see; we don’t know how to fix it yet. What are some of the options, or do we just not go forward with that because the roadblocks are there and we’re not sure how to how to fix it. So, I think it allows Amazon to be much richer in the beginning, and forces people to think deeply about what they want to accomplish, not just kind of picking a goal out of the air, going “That would be nice, but how are we going to do it too?” So, I think that may be a nuance there that Amazon has added to it.
Tim Meinhardt:
Yeah, terrific, and wonderful insight. So, Steve, we’ve been talking here primarily about input, and, you know, I’m going to close with one final question, and maybe we can bring it around the corner is, and again, I probably know this answer, but I would love to hear it in your own words was, why is Amazon so focused on data and measurement?
Steve Anderson:
It’s an interesting way to phrase that question. You know, my first thought is I think Bezos, and obviously now the team at Amazon, understands that data helps you gain insight. And I’m going to bring it back to a question I ask all the time in my presentations. Have you bought something on Amazon in the last 60 days? You know, hands may go up or not. In fact, I did it just recently, a couple of weeks ago, and I said, “Anybody buy anything from Amazon and have it delivered to the hotel?” and one hand went up.
Tim Meinhardt:
I love it.
Steve Anderson:
And you know, but then the question is why? And almost always the answer is because it’s easy, at least one of the first answers, right? And so, I say that because Amazon one has spent years and billions of dollars making it easy, and a lot of that, if not most of it, is because of the data they collect on their customer, not just to analyze purchase. They do that history and write all those kinds of things, but also to create in their website experience a personalized experience for every single customer. So, I think they understand Bezos, and again, the team understands, that the goal is customer obsession, you know, making the customer have as frictionless as an experience as is possible, and data helps them do that and helps them understand the customer and how to deliver that better. And that’s kind of one example, you could look at their marketplace sellers, they’re in all kinds of different parts of the Amazon ecosystem, are all part of the data. But it’s not just data for data, it’s data for improving the experience of their various customers, be they AWS or marketplace sellers or consumers or whoever is interacting with Amazon.
Tim Meinhardt:
Right, wonderful. Because, you know, you kind of look at what’s really most important? What is it at the end state? What is it really? It’s customer focused. What does our customer really want? And then you begin to track all of the things that that drive you, to that that basically that simple one or two concept experience. Gosh, Steve, thank you so much again, everybody, the book is The Bezos Letters. It’s by Steve Anderson. It is a fantastic read, and I would encourage everybody to pick up a copy of this and read it not only over the holidays, but whenever you want because it is an excellent book. Steve, it is such a pleasure to have you back on the program, as always, my friend.
Steve Anderson:
I appreciate you inviting me back, Tim. So, it’s been a great conversation with you.
Tim Meinhardt:
Yeah, this is great. This was great. So, with that, Steve, I hope you have a wonderful day. And again, we’re going to have you back again, Steve.
Steve Anderson:
All right, sounds great. Thank you.
Tim Meinhardt:
You got it, goodbye.
Thanks so much for taking a few minutes to listen to our OKRs Q&A Podcast. You know, OKRs provide such an excellent, agile framework which is critical for today’s business needs. It’s such a pleasure to have such wonderful people share their stories and journeys with us. Please, should you ever need assistance with your OKR journey, do not hesitate to reach out to us and contact us at www.atruity1.com, and make sure if you have a minute, to rate our show. Have a great week. Stay healthy. And of course, stay happy. Thanks, everyone.
