Devil (un)disqualified decisions: only move ahead with decisions that the devil cannot disqualify (aka the work needed to build an opinion)
/By Duncan Anderson. To see all blogs click here.
One sentence summary: To build confidence in a decision you first need to get to a point where you believe the decision is makes sense for what you are trying to solve through standardard problem solving methods, then you need to systematically work through all the reasons why the decision does not make sense, every time you disqualify a reason why the decision does not make sense will gain more confidence in your decision.
Blog summary:
The work needed to build an opinion:
1. Try to find all the reasons why a decision will work and subsequently validate / invalidate the reasons. Each reason to move ahead with a decision you can validate increases the probability to go ahead.
“Why something makes sense”
2. 10x thinking. eg How can we make 10x the revenue next year, how can we take 10% the time to build a Mustang for next year, etc.
I find that massively moving the goalposts prompts serious creativity, it’s so easy to get stuck with incremental thinking :(.
3. Try to find all the reasons why a decision won’t work and subsequently validate / invalidate the reasons. Each reason to not move ahead with a decision you can invalidate increases the probability to go ahead with the idea.
“Why something doesn’t not make sense”. aka playing Devil's Advocate.
Jingle time: “before you can move ahead with a decision you need to be able to tell the devil to f$%^ off!” AKA devil (un)disqualified decisions :).
Want to end up in heaven? First make sure you are not headed to hell!
4. Run through any second and third order consequences (link for details) from your decision
Things to note:
Not all reasons are born equal. It doesn’t matter how many reasons you have to go ahead with a decision, one deal breaker means you shouldn’t do it.
Not all decisions are born equal. Some have more consequence than others (eg small / medium / large). For medium+ decisions I don’t believe you should move ahead unless you have been able to tell the devil to f$@# off.
Graphical representation:
You start at no confidence in a decision.
Then you try to improve confidence by finding reasons why the decision makes sense.
At some point it the best way to further improve confidence in a decision is to try to ‘disqualify’ the decision by playing Devil’s Advocate. If you have tried your hardest to disqualify a decision but it still makes sense to go ahead then let’s do it :).
Should each blog have a poster? Why not?
Something about giving the finger to the devil seems about appropriate to me.
Please someone come up with a concept, let me know and we’ll make a poster out of it.
++++++++++++++++++++++
Primer info:
What are strawmen and steelmen arguments?
Strawman:
A strawman is a version of an argument that no one actually believes, but is very easy to dispute.
Steeleman
In contrast, a steelman argument is the strongest version of an argument, sometimes called the principle of charity. The goal of steelmanning is to avoid attributing irrationality, logical fallacies or falsehoods to others' statements, when a coherent, rational interpretation of the statements is available.
Using a strawman vs steelman argument
If you just wanted to win the debate (and annoy the person you’re speaking to), pick a strawman of their argument and burn it to the ground. If, however, you want to actually try to get to the truth, you should consider the strongest possible argument that your opponent could have meant and reply to that.
Giving the devil his due (aka playing devil's advocate)
Trying to look at the other side of an argument and seeing all the reasons why something might not work (or might work).
Confirmation bias
Confirmation bias is the tendency to search for, interpret, favor, and recall information in a way that confirms one's preexisting beliefs or hypotheses. It is a type of cognitive bias and a systematic error of inductive reasoning.
Confirmation bias is also the tendency to disregard information that denys one's preexisting beliefs or hypotheses.
Charlie Munger on the work needed to have an opinion (Charlie Munger is one of my heros).
“I never allow myself to have an opinion on anything that I don’t know the other side’s argument better than they do.” — Charlie Munger
“We all are learning, modifying, or destroying ideas all the time. Rapid destruction of your ideas when the time is right is one of the most valuable qualities you can acquire. You must force yourself to consider arguments on the other side.” — Charlie Munger
“The ability to destroy your ideas rapidly instead of slowly when the occasion is right is one of the most valuable things. You have to work hard on it. Ask yourself what are the arguments on the other side. It’s bad to have an opinion you’re proud of if you can’t state the arguments for the other side better than your opponents. This is a great mental discipline.” — Charlie Munger
Other quotes
“The ability to change your mind is a superpower.” Ray Dalio.
"To be absolutely certain about something, one must know everything or nothing about it." — Henry Kissinger
“Self-belief must be balanced with self-awareness. I used to hate criticism of any sort and actively avoided it. Now I try to always listen to it with the assumption that it’s true, and then decide if I want to act on it or not. Truth-seeking is hard and often painful, but it is what separates self-belief from self-delusion.” Sam Altman
Details baby :)
For the purpose of this blog I’m going to simplify the world into two types of decisions:
1. Where you know what the answer to a decision should be
2. Where you don’t know what the answer to a decision should be
I’m just going to talk about decisions where you don’t know what the answer should be today.
What have I got for ya?
Decision making = 1. Determining the level of confidence needed to move ahead with a decision + 2. Theoretical exploration of a decision + 3. Practical exploration of a decision
Decision Journal Framework
An example
1. Determining the level of confidence needed to move ahead with a decision
When you are trying to make a decision where you don’t know what the answer is you need to figure out what the appropriate confidence threshold (confidence sufficiency) is before you can go ahead with the decision.
Where you do not know the answer you cannot be 100% certain so you need to determine your confidence sufficiency threshold (aka uncertainty threshold) where you are comfortable move ahead with the decision. However, the level of sufficiency should differ depending on the circumstances of the decision.
Eg, do I need to be 50% certain, 75% certain, 90% certain to go ahead.
Here is the main framework I use to figure out what the confidence threshold is:
Confidence threshold = 1. How easy is it to reverse the decisions + 2. How much work is involved in a decision
Reversibility: some decisions are easy to reverse, some are very very difficult. As such you should be more careful before going ahead (ie higher confidence threshold)
Work involved: some decision have heaps of work involved, some very little. All else equal the more work involved the higher the confidence threshold should be.
Guidelines for low, medium and high confidence thresholds:
Low = 50% confidence it is the right thing to do:
I typically don’t move ahead with anything I don’t have at least 50% confidence in.
You’ll see below that sometimes the best way to increase confidence is through ‘practical exploration’ of the idea.
Ie i have 50% confidence from ‘theortically investigating’ the idea and as it’s low risk the best way to get more confidence is to ‘practically investigate’.
Medium = 75% confidence
You have investigated the decision theortically and have been able to do some practical investigation but due to the scope of what needs to be done you aren’t able to exhaustively investigate all avenues
High = 90% confidence
If I don’t know the answer I don’t think it is really possible to have more than 90% confidence until we have actually tried something (ie max confidence at ‘theoretical’ stage is 90% confidence, can only get to 100% confidence once you have ‘practically’ tried something).
2. Theoretical exploration of a decision
Theoretical exploration of a decision (idea) is seriously something, I think, I could write a book on.
I only wanted to talk about one part of this today which is ‘the job of playing devil's advocate’. However as usual I’ve managed to massively blow out this friggin blog :(.
What I would normally do?
Write a proposal centered around ‘universal problem solving framework’. What is the ‘universal problem solving framework’?
1. JTBD = write down the job to be done
2. MECE the JTBD = Mutually Exclusive and Collectively Exhaustive break down the JTBD from at least one lens
3. Make a model of ‘2’ = turn 2 into an equation and model the variables out through eg a taxonomy
4. Crunch the model from ‘3’ = run a few scenarios through your model to see where it help, hinders and breaks and adapt the model accordingly.
5. Synthesize learnings from ‘4’ into a report.
After this I’d send the report from ‘5’ to an appropriate group of people for them to read, digest and come up with questions before we have a meeting to discuss it.
What happens in the meeting where we discuss the report?
1. We decide on the ‘confidence threshold’ to move forward. Let’s say that it’s 75% for this decision.
2. We then put forward reasons in turn why we think the recommendation will work and debate them as a group to see if they are valid. Each point we think is valid should increase our confidence to move ahead with this decision.
3. Practice 10x thinking. Eg how do we make 10x the revenue of this year? Eg how do we take 10% of the time to build a mustang vs last year. I find that forcing yourself to think big like this typically generates ideas you otherwise wouldn’t have gotten!
4. Once we get halfway to the confidence threshold (eg ~40% confidence we should move ahead with this decision) we then switch from putting forward ideas of why this will work to putting forward idea of why the proposal won’t work (ie giving the devil his due). Here we try to put forward all ideas for why the decision is bad and then we debate. Ultimately any idea we can put forward that we can’t see stopping the decision is a way to ‘increase’ confidence we should move ahead.
However it is important to remember: “It doesn’t matter how many how many dealmakers you have for a decision, just one dealbreaker means you shouldn’t move ahead.”
5. Try to look for any second and third order consequences from the decision. I find it’s often easy to look narrowly at the outcomes of your proposal and not consider who else it might affect.
5. If we get the confidence threshold them we move ahead!
Graphical representation of the discussion in the meeting:
It makes no sense to look at why you should not do something until you have some confidence it makes sense
Afterwhich, any reason we can find that not move ahead that we can disqualify is a way to increase confidence.
It’s really important to try and look at a problem from both sides and not ‘get caught up with confirmation bias’ etc.
Aside: I find that most people have a pessimistic ⇔ optimistic set point.
Ie they are typically up one end more of the continuum more than they should be. Systemically going through why to do something and then why not to (playing devils advocate) with a group I find is a great way to overcome your innate biases and also everyone knows they will get the chance to put forward reasons why something won’t work.
This I’ve found makes for much more energising meetings :)!
I find that problems come when you have an incorrect view of reality, ie you neither want to think your idea is better or worse than it actually is (overconfident / underconfident).
One key way I try to get around this is by systemically with a group trying to see why an idea will work and then group trying to break an idea (playing devils advocate) and then asking each person what they think the confidence level this is a good idea is.
3. Practical exploration of an decision
Finding through doing
You can only gain so much confidence through talking through an idea theoretically, at some point you have to put rubber to the road.
You almost never have full information about making a decision… and sometimes the easiest way to get more information is to make the decision and see how things go.
If a decision is easy to reverse than try out the proposal and see where you are wrong (eg didn’t have full information or had errors in your synthesis). I find ‘practical’ knowledge is often the fastest way to learn vs sitting around trying to gather more information ‘theoretically’.
Aside: I think in the past I’ve spent too long discussing an idea vs just trying it out to see what is what.
Even if a decision is hard to reverse one hack I like is just to ‘run a pilot’.
Eg you don’t need to build an entire Mustang and ship it to schools to get a unit of learning. You can just make 1x lesson and then show it to 5x students and 5x teachers to get feedback. If the feedback is good then you build the idea for the entire Mustang, if it’s not good then you learn why and adjust the idea and then run another pilot.
Usually we get these learnings by implementing a batch size of 1 (BS1) to investigate the proposed solution
The reason to do a BS1 are to
Test whether a solution will work
Find issues before scaling the solution
Both reasons are centred around exploring a decision and finding learnings so you can be confident in what you are proposing to do
Aside: this is also called ‘lean methodology’. Build the MVP (minimum viable product), then test it (measure) and then learn. Then update what you want to build. Build => measure => learning => build etc etc.
Decision Journal Framework
Want to level up decision making even further? Then try having a decision journal! FYI I have one of these.
What is a Decision Journal?
A place where you note down your decisions and then come back in the future and review your decisions to systematically learn.
DA’s suggestion of what to include in a Decision Journal:
What is your confidence sufficiency threshold to move ahead and why?
Is this decision easily reversible or not?
How much work is needed to actually get a unit of learning? Can I get a unit of learning with 10% of the work (eg a pilot)?
What percentage certain am I that this proposal is right?
What are the key reasons I think the proposal will work?
What are the key reasons I think the proposal won’t work?
Have I carefully looked for any dealbreakers to make sure there are none? (it doesn’t matter how many dealmakers there are, just one dealbreaker will cause this to fall over).
Have I stress tested this decision with an appropriate number of other people?
What is my headspace when making this decision (eg confidence, not confidence, relaxed, stressed, etc)?
Date I made this decision?
Date I should review this decision?