All models are wrong, some are useful
/By Duncan Anderson. To see all blogs click here.
Reading time: 11 mins
Summary:
Almost always the world is more complex than one binary variable.
Often the world is infinitely complex.
What to do?
IMO build a model of 2-5x grey variables that explain 80%+ of a problem space.
IMO the right 2-5x variables normally explain 80%+ of a problem space.
IMO normally your mind can understand and flex a 2-5x variable model… I find that 10x variables is almost always too much!
IMO if you don't have a model to explain a problem space, it's likely you will have large blind spots and ego distortions for that problem space. This means you can often think you are progressing when in fact you are regressing!
The solution to all problems is understanding. One core method I use to improve ‘understanding’ is upgrading models to better explain problem spaces.
I believe the best problems are infinitely complex: eg what does it mean to live a good life, what is the common good, how to be a good friend, how to run a government, how to improve education…
Whether you are aware of it or not there is always an existing solution to a problem space… even if the solution is ‘there currently is no solution’.
For the most complex problems I believe you can likely always improve the solutions, one approach is building generations of upgraded solutions. So I try not to worry about ‘what the perfect solution is’ but ‘is this solution better than the existing outcome’.
IMO one of the most important skills one can cultivate is the ability to build ‘useful’ models that explain problem spaces.
Useful model = 1. A model that has 80%+ explaining power of the problem space + 2. Is simple enough that you can manipulate the variables (either in your mind or eg a spreadsheet) AND understand the overall changes, changes in one variable make on the model.
I find building models to try and improve my understanding of the world is hard… but oh so much fun.
IMO for the major areas of your job you should consider having and constantly upgrading models that explain the problem spaces in which you operate.
One lens: Input * Model = Output
People often give feedback on output, or say try harder on input. I believe that often the optimal layer for which you can understand how someone is thinking, and for which someone can upgrade is by grasping the model they are using to understand the world. To sustainably improve output (to teach people to fish, not give them a fish), I often find the best approach is to understand and help improve the model someone is using.
Jingle: all models are wrong, some are useful.
Models, skinny things you hang clothes on or ways to improve the world?
In some respects, IMO your ability to problem solve / innovate = your ability to create useful models.
In some respects, IMO your ability to improve your own understanding (vs others improve your understanding) = your ability to create useful models.
*Examples of models are at the bottom of the blog.
++++++++++++
Details
Is the world just a series of decisions?
One lens for life: a series of decisions.
If life can be viewed as a series of decisions.
And decisions done well require problem solving.
And problem solving is based on an understanding of a problem space.
Then your ability to make models to explain problem spaces is key to living a good life.
One lens for businesses: the accumulation of a series of decisions...
IMO if you work at a restaurant you need to make decisions like the menu, hiring etc
IMO if you are a teacher you need to make decisions like how to give feedback, what activity to do, etc
IMO if you are doing a startup you need to make decisions about what your product should be, how to market your product, etc
One articulation: Decision = 1. Understanding of problem space * 2. Synthesis of possible solution sets
IMO building useful models that explain problem spaces => one key way to improve at “1. Understanding of problem space”
IMO building useful models that explain problem spaces => one key way to improve at “2. Synthesis of possible solution sets”
If much of life is ‘decisions’, then IMO one core way to a better life is to improve at ‘building useful models that explain problem spaces’.
IMO the CloudStreaks blogs are a ‘model + words’
IMO the Edrolo products are ‘model actualised into eg a textbook’ (ie a recipe * a machine)
IMO feedback to someone is a ‘model actualised into an upgrade opportunity’.
I don’t think I used to be able to see the world this way, I think 5 years ago I was ‘just trying to do good’, not ‘systematically trying to do good through building models’.
IMO if you can’t explain what you are doing in eg a model it’s very unlikely you understand what you are doing. IMO if you can’t explain how to do something well or how to do something badly you likely don’t understand what you are doing. One core way I use to try and explain what I’m doing, what is good / bad is through models :).
IMO for all core parts of work one should have models to compliment explaining what you are doing.
IMO your ability to create models to explain problem spaces in some respects is your ability to change the world.
Models, don’t scout them, build them ;).
Strategic thinking stages V1 - IMO almost never over simplify the world into ‘one binary variable’
What I find:
Typically the most important variable will have 30-60% explanatory power of a problem space and needs 3-5x levels in a taxonomy (see taxonomized thinking).
The 2nd most important variable has 20-40% explanatory power of a problem space and needs 2-5x levels in a taxonomy.
The 3rd most important variable has 10-30% explanatory power and needs 2-4x levels in a taxonomy.
Variables 4 & 5 are 10-20% explanatory power and need 2-3x levels in a taxonomy.
Comment
Figuring out what these variables are and then creating taxonomies for each variable that are ‘useful’ (ie have high explanatory power) I find super duper hard… but is the core approach I’m aware of for making better decisions.
Strategic thinking stages V2:
-L2: argue not for what is the ‘overall best solution’ but whatever your first view point is. Changing your mind is a super power. If you want to be ‘right’ often you need to change your mind often.
-L1: oversimplify a problem space to one binary variable and say that because of this variable it’s either a good / bad idea.
Honestly I feel that this is the default way people put forward ideas in verbal discussions until they are trained up.
IMO this is one articulation of a Strawman Argument.
If someone is doing this, I say ‘please build me a multi variable model to explain the problem space’. Otherwise you can have one party talking about Variable A and the other about Variable B and both thinking the other isn’t listening… when it’s that both are doing low level strategic thinking.
L1: can understand multiple variables and discuss the pros and cons of them vs each other trying to balance.
L2: can create taxonomies for variables that have high explanatory power.
L3: can figure out the 2-5x variables and accompanying taxonomies that give 80%+ explanatory power of a problem space.
L4: can manipulate the variables in relation to each other understanding how changes in one variable affect other variables.
Comment:
IMO it’s not ok to argue for your position vs the overall right thing to do.
IMO it’s not ok to simplify to the point of absurdity.
Typically one is going after a solution better than the existing outcome. IMO for the most complex problems (aka the best most fun problems) there likely is never going to be a ‘perfect solution’, but hopefully slowly improving generations of solutions.
IMO don't think about if the idea is right or wrong, think about how the idea might be useful… and how you can layer this into a high explanatory power model.
All models are wrong (ie not explaining everything), some are useful (ie help explain more than you previously understood).
IMO you are trying to build a multi-coloured 3D model, not a black / white 1D piece of poop!
Modelling: one key way to create beauty in the world ;).
“A Posteriori” vs “A Priori” models
A Posteriori = a model created from existing data points. So you have a set of information and build a model to explain the data.
A Priori = a model created to explain the world before having data points. You then run scenarios through the model and see how well they explain outcomes.
Models almost always need to be calibrated to reality. But IMO if you can’t build models ‘A Priori’ you are going to be moving way slower. The key way I know how to get better at building models is to… build models AKA deliberate practice.
I’m almost always building models to explain ‘thought’, not building a regression based on actual hard datasets. In some respects you are trying to get a high R2^2 value but you can’t ‘know’ like eg if you are doing regression analysis on actual data.
Explaining thought (eg how to give feedback, eg how to make high quality exam style questions, eg how to build self esteem through textbooks) normally means you can’t have ‘hard data sets’ to actually ‘regress’, but through building a model with high explanatory power a priori IMO one can ‘progress’.
+++++++++++
Example: Secondary Textbook Questions - A Posteriori
This is an example of a model built to try to explain ~80% of the variation of past Exam Questions.
Question = 1. Theory + 2. Question Information + 3. Task Word + 4. Marks
This had a far more complicated taxonomy for ‘3. Task Words’ than I’m generally talking about. Yes some of the models we use are way more complicated that I’ve alluded to here. Bah!
Comment
Through this model we believed we could understand ~80% of what made up exam questions and as such make high quality new exam style questions.
Edrolo only reference: one articulation of a collection of models are the content recipes and content machines.
Example: types of teachers - A Priori
MECE of teacher types:
1. Out of area (teaching a subject they don’t know well)
2. Traditional
3. Hard working Traditional
4. Innovator
Comment
You are unique… just like everyone else. Your personal experience is fair and valid, but your personal experience does not represent the entirety of human experience.
The more you can empathize with the people you are trying to help the more you should be able to help.
While each teacher is different, if you segment up the teacher cohort intelligently into 2-5x ‘personas’ you should be able to have ~80%+ understanding broadly of what the full cohort of teachers want.
Then when building products you try to understand how each persona sees the world and explain how the different personas will respond to the parts of your product.
One variable we carefully consider at Edrolo: by switching to Edrolo from an existing product will a teacher persona get back time or need to spend more time.
Example: feedback - A Priori
Feedback = 1. Specific (the other party understands what you are referring to) + 2. Upgrade opportunity + 3. Tone
1. Specific - Taxonomy
Bad: the other party doesn’t know what you are referring to
Good: the other party properly understands what you are referring to
2. Upgrade opportunity - Taxonomy
Good: not just this was done well or done badly, but exactly how to improve in an understandable and implementable way.
Bad: this output was good / bad. Please do better next time.
3. Tone - Taxonomy
Bad: negative sum dressing down that makes someone scared to bring ideas to you
Good: positive sum we can all improve at almost everything always, we improve best if we help each other.
Example: life - A Priori
Model: MECE of time = 5 days of purpose, 1 day of play, 1 day of peace.
Purpose: Find a way to make the world better (eg improve education) => take on responsibility to affect change => get meaning => are happy. Or Purpose = Fun * Consequence (responsibility)
Play = Fun * No consequence (no responsibility)
Peace = Do nothing (not having ‘fun’)
Comment
Understanding that I want different types of time in my week and that doing each ‘well’ can be totally different took me a while.
I believe to work well, one should rest and relax well.
IMO one can’t try to relax, it’s a paradox. But IMO one can and likely should try to work well.
So in some respects, the modus operandi for working well is the exact opposite modus operandi for relaxing well.
I personally don’t think I’m that good at relaxing. For 2021, I think one of the key ways for me to work better is to relax better. 5 years ago Duncan was all ‘how do I improve my productivity’. Today Duncan thinks he’ll improve work productivity by doing relax time each week better. Relaxing time = zero productivity. So… for Duncan to improve work productivity = get better at zero productivity :)
Examples from my blogs - I try to have a model is every blog
Decision frameworks: “How to approach solving problems is itself a problem to solve.”
Helping Humans Handbook - don’t just support, also push, intervene and leave alone!
Examples from others (there are heaps)
Bloom’s Taxonomy
Maslow’s hierarchy of needs
If you only take away one thing
To get better at understanding the world get better at building A Priori models that explain the world.
I honestly believe you can make models to help explain almost all parts of life. If you don’t have a model for an important part of life, I’ve found it’s really easy to miss the wood for the trees.
Want a model life? Model out life!
IMO improving one's ability to a priori create models for thought problem space is one of the key ways to improve at everything!
++++++
Addendum: I couldn't help myself
“In a comprehensible universe, if something isn’t forbidden by the laws of physics, then what could possibly prevent us from doing it, other than knowing how? In other words, it’s a matter of knowledge, not resources.” David Deutsch
Engineers today are biologically indifferent to those 200 years ago but today we build iPhones and rockets to Mars. What we can do is literally other-worldly.
We build models for things like atoms (electrons, neutrons, protons etc) and this allowed us to build things like computer chips. Then computer chips allowed us to build rockets with very intricate designs. Now we can send things to Mars!
So knowledge creation ability = model creation ability?
So as above, model creation ability = problem solving / innovation ability? Maybe!!!