PB 13/10: Don't look back in anger
How Pre-mortems, Corrections of Error and AAR's help you make sense of the past (even before its happened) and make better decisions in the future
I wrote last week about making decisions - something I’m going to come back to in future weeks. This week I want to dig into the world of pre-mortems and post-mortems.
There’s a connection: systematically anticipating what might go wrong, and learning from what’s gone either right or wrong are also critical parts of any successful decision-making culture.
All are great practices that are worth investing in. All are as much from the project management world - but that doesn’t make them any less relevant to Product Managers - especially when a big initiative suddenly lands in your lap.
Pre-mortems: because prevention is better than cure
We all love a good motivating kick off to a project. Getting everyone in a room (or on a Zoom call), and talking about the great stuff this awesome team is going to do.
The problem with all of that relentless positivity is that people don’t talk about the stuff that’s bugging them and totally avoidable errors happen.
Pre-mortems are a way round that. Essentially getting everyone in a room (or on another Zoom call), doing a future retrospective and imaginging all the things that might go wrong and then agreeing a plan for how to avoid this.
It’s rather like Scrooge’s encounter with the Ghost of Christmas Future.
If you want to know how to actually run a pre-mortem, Shreyas Doshi has a great template from his time at Stripe that can simply lift.
In this he talks about three types of issues/ potential problems.
🐯 Tigers - A clear threat that will hurt us if we don’t do something about it.
📃 Paper Tigers - An ostensible threat that you are personally not worried about (but others might be).
🐘 Elephants - The thing that you’re concerned the team is not talking about
But if you want something a bit less fancy, you can always look at this version from Riskology (’Leadership for Introverts;), which basically says
Get everyone in a room
Spend a chunk of time listing every possible problem - then picking the top 10
Then work on your solutions for those top 10
It matters less exactly how you do a pre-mortem; and more that you do one in the first instance. If you want to try it - start by doing it with a personal project: a kid’s birthday party or your next holiday. It’s remarkable how good it feels to plan from a pessimists perspective.
When it all goes wrong: Amazon’s CoE machine
But, even with pre-mortems, stuff goes wrong. And one of the things that differentiates great companies from OK companies is how they react when it does.
One of the practices at Amazon that doesn’t get written about a whole lot is the Correction of Error process. This doesn’t, for example, get the same amount of coverage as writing PR/FAQs for product launches, but it’s just as pervasive a part of the culture.
The basics of the process are similar to error/ disaster review processes in lots of places.
When something goes wrong (normally a technical or operational error) the team writes a Correction of Error within 14 days.
As well as detailing the specifics of the error - you ask 5 ‘Why’s’ to get to the root cause of the problem. You’re looking to see where your blind spots were.
Once it’s clear what the root cause is, you define a set of corrective actions to make sure it won’t happen again.
That bit is pretty straightforward, and I’ve put links to a couple of CoE templates below (this from Working Backwards is the closest to the original); but there’s a few aspects that are very ‘Amazonian’ and turn this process from being a simple source of good intentions to a genuine mechanism for improvement. (note: This is how the system worked when I was there - it might all have changed since then!)
Every CoE exists on a single CoE site. You submit it, and it stays there permanently.
Anyone within Amazon can read any CoE (a very small number are kept private) - and if you want, you can subscribe to an email list so every CoE arrives in your inbox.
A number of VPs from across Amazon continuously scan the CoE queue and will ask questions / seek clarification if needed.
Your corrective actions have dates and owners on them, and are tracked centrally. You will get a mix of automated reminders if you don’t follow up, which are eventually escalated to your manager.
So it’s not just the CoE itself. The mechanics of Peer review ensured that people really got to the bottom of their problems and actually acted on them. Having the single database and feed provided a constant source of learning for the teams.
CoEs never point the finger - they’re not about allocating blame. They’re about understanding system faults and fixing them. Knowing that the process exists means that if you hit some crisis, everyone can focus on dealing with the crisis right now, and that the CoE will follow to make sure this doesn’t happen again.
Some of the CoEs on the system were legendary. The one that looked at Amazon’s near meltdown after offering a Lady Gaga album for 99c (called, I seem to recall, ‘Gagging on Gaga’) was required reading in CoE class (no, there wasn’t an actual CoE class!).
My personal CoE moment followed the launch of Prime Video in the UK - where it wasn’t possible to find any titles in the main Amazon website search. Complex, high octane, and crossing multiple teams - but the outcome was a playbook to ensure that was a one-off error.
When you move fast and try ambitious things, stuff goes wrong. That’s not a problem. What matters is understanding why stuff goes wrong, facing up to it, and making sure it doesn’t happen again. Each one of the thousands of CoE’s on file resulted in Amazon doing something better next time.
AARs: because sometimes stuff goes well, too
Of course, everything you work on isn’t either a disaster, or a disaster about to happen. Every so often stuff goes really rather well or even just OK.
The After Action Review seems to have been developed in the US Army as a standard post-mortem for every action, and the model now gets used widely for all sorts of activity (sample template here from Asana - with others below, including one for the NHS). It’s similar to a sprint retrospective. The trick is to be just as challenging and analytical when something goes right as when it goes wrong.
When Clive Woodward was coaching the England Rugby Team he had a mantra of analysing success much more than failure, as paraphrased here..
When you loose, take everyone out to the pub. When you win, analyse in detail HOW you won. Break down and analyse your "winning moves", then codify these so you can practice them and get even better.
Well - perhaps not the pub when everything goes wrong, but the point is if you want to make sure you’re good, not just lucky, it’s worth diving into success at least as much as failure.
The loop back to better decision making
I started this by saying there’s a link with my recurring theme of decision making. Your decision making ability - as an individual and as a team - is like a muscle, it gets developed over time - and all of these exercises are highly effective workouts.
Often what you find are mistakes in process rather than product sense: involving the wrong people at the wrong time in the wrong way; not sharing information; either massively over-estimating or under-estimating the take up of something due to missing or mis-interpereted data. All of these are effectively decision making steps.
The experience of what you learn in a CoE feeds into your next pre-mortem. Which will help you make better calls when you’re even before the pre-mortem stage and thinking about your next wizard scheme. Even better if all of the documentation from these is shared and made easily accessible (as with Amazon’s COEs) so that everyone can benefit from the knowledge.
If you doubt they’d work at work - just try implementing at home. I’m sure the kids will love an AAR after their next spelling test…
Resources
Shreyas Doshi: Pre-mortem template
Joshua Harris: CoE Template
Working Backwards: CoE Template
Gary Klein: Performing a Project Pre-mortem
Riskology: Pre-mortem technique
Asana: After Action Review template
NHS: AAR template
Toggl.com: How to conduct an after action review
If you’ve found this useful..
Feel free to buy me a coffee. It lets me know I’m heading in the right direction!