Diagnose before you prescribe. Quality diagnosis = 100% Coverage Problem Space framework = Building a framework to explain ~100% of the problem space.
/By Duncan Anderson. To see all blogs click here.
Reading time:
Intro = 4 mins
Details = 7 mins
Summary: For large problems I believe one should have a framework that covers 80%+ of the problem space AKA ~100% Coverage Problem Space Framework
Jingle: Diagnose before you prescribe. For a perfect presecription don’t have a dumb diagnosis.
You can’t have a good solution if you don’t understand the problem space properly unless you get lucky.
One key strategy I have to ‘diagnose’ well is to build a framework that covers at least 80% of a problem space. AKA 100% Coverage Diagnosis Framework.
Some suggestions literally make things worse
Rearticulations
The path to hell is paved with good intentions.
The path to hell is a prescription based on a poor quality diagnosis.
The path to hell is paved with good first order intentions and unforeseen negative second order consequences that more than offset the first order improvement.
The path to hell is optimising for a portion of the problem space picture (eg 25%), and not realising that this optimisation actually means going backward for the other 75% of the problem space picture for an overall net loss.
The path to hell is prescribing without a 100% Coverage Problem Space Framework.
Diagnose before you prescribe
Diagnose = Build a framework that covers 80%+ of a problem space AKA the full picture of a problem space with metacognition explanation AKA 100% Coverage Problem Space Framework
-L2 = no piece of the problem space picture
-L2 = a piece that isn’t part of the problem space
-L1 = one piece (eg 25%) of the problem space and think it’s the entire problem space
L0 = a picture of that represents part of the full problem space, but not the entire space (eg 50% of the problem space)
L1 = a framework that represents 80%+ of the problem space
L2 = L1 + have a meta-explanation of the problem space that makes it easy to understand and work with AKA a 100% Coverage Problem Space Framework
Prescribe
-L2 = go with the first thing that comes to mind.
-L1 = come up with two options and calibrate them vs “-L1 = one piece (eg 25%) of the problem space and think it’s the entire problem space”
L1 = come up with 2+ options and calibrate them vs “L2 = L1 + have a meta-explanation of the problem space that makes it easy to understand and work with” explaining the weighted best outcome and tradeoffs.
Reference blog: Externally supported recommendations, not opinions
Comment
This blog is mainly about trying to figure out minimum sufficient diagnosis, not about levelling up prescription.
With the benefit of hindsight I think for years I underestimated the importance of diagnosis. AKA operated with sub sufficient diagnosis.
Outcome = Diagnosis ability * Prescription ability
Anything times zero is zero.
Diagnosis is upstream of prescription.
I see this blog as a levelling up of ‘To solve problems effectively, first build a complete picture of the problem space.’ The 100% coverage part and frameworks to show this is the main level up.
Examples of headline ‘~100% Coverage Problem Space Frameworks’
Example 1: People management
Areas of the problem space:
Area 1: the individual
Area 2: the team
Area 3: the manager
Area 4: the company
Most of the time you can have all 4 areas be happy, but normally not all of the time. For example, sometimes there is one person who is underperforming and it can feel harsh to performance manage them. But when you look at the ‘100% Coverage Problem Space Framework’ you can see the ‘line of sufficiency’ to performance manage someone is well and truly met.
Not performance managing someone is actually, while possibly in the interest of the individual (ie one piece of the problem space), a net loss for the full problem space according to the ‘100% Coverage Problem Space Framework’.
Example 2: Making education products
*Note: most of the rest of this blog will be concentrated on this area as this has a lot to do with what Edrolo does.
100% Coverage Problem Space Framework = 1. Framework Destination + 2. Optimal starting point + 3. Systematic sequential path between the startpoint and Framework Destination
While there might not be a ‘100% definitive’ destination, not doing the work to build an agreed upon ‘fuzzy desintiation’ (often previously referred to the ‘fuzzy north star’) means any path is ‘forwards’... also any path is ‘backwards’.
Having a 100% Coverage Problem Space Framework I find core to being able to put forward an ‘Externally Supported Recommendation’, not an opinion.
More details below.
++++++++++++
Details
100% Coverage Problem Space Framework = 1. Framework Destination + 2. Optimal starting point + 3. Systematic sequential path between the startpoint and Framework Destination
This framework was built for Year 7-10 Education products but I think can be used in other areas too.
Building the 100% coverage diagnosis frameworks can be really hard, but not doing so is likely dooming yourself to failure :(.
Most of the rest of this blog is on this framework.
Framework Destinations - some are easy to figure out, some are hard, however I think having one is a necessity
Tesla - easy: cheaper electric car or more specifically cheaper average cost per kilometre travelled.
Google search - hard: better quality search
One of my fav presentations from when I was at Google back in 2011/12 was how Google had made a quantitative measure for Google search quality. They ‘whitelabelled’ the major search engines (ie removed discernable UX elements) and then got external people to search on the different players (eg google, bing, yahoo) and rate their outcomes. You could see the quality number on average of Google improving and the gap to Bing increasing.
There are a huge amount of semantics involved. Eg if someone googled ‘streaming platform’ 20 years ago what did it mean? Streaming = water, platform = ?? maybe a waterfall. Now? Netflix.
For Google Search the Framework Destination is constantly moving but it is able to be mapped.
Secondary education resource - medium: 1. Find the types of things students need to do (eg exam, eg inquiry project, eg socratic discussion, etc) * 2. Map these done well at high resolution (in Edrolo speak ‘genome’) * 3. Munge together
Figuring this out takes lots of work. But it’s crucial.
Bridgewater country rankings - hard:
This is part of the way Bridgewater build a ‘100% Coverage Problem Solving Framework’ for investing.
Full file here.
You’re not listening to me, no you’re not listening to me!
I find that when someone says something there is almost always a supporting reason. There however isn’t necessarily a supporting ‘100% Coverage Problem Solving Framework’ through which they are calibrating multiple options and considering multiple variables. AKA someone has done sub sufficient diagnosis before prescribing.
A recommendation isn’t necessarily a good one as you have one supporting reason. In fact the proposal could be an improvement on the one variable (piece) of the problem space put forward but overall highly counter productive for the problem space you are operating in.
Levels
-L2: opinion with no supporting reason
-L1: opinion with one supporting reason
L1: opinion with discussed and agreed up high resolution Framework Destination
L2: externally supported recommendation based on a 100% Coverage Problem Space Framework
L3: L2 + calibrate existing proposal vs new proposal on the 100% Coverage Problem Space Framework with the multiple variables this entails.
An oversimplification: you can have negative sum debates or positive sum discourse.
Often negative sum debates happen when two parties are operating at ‘-L1: opinion with one supporting reason’. AKA don’t have a 100% Coverage Problem Space Framework AKA both have only one piece of the problem space and the pieces are different to each other.
My supporting reason is good, no my supporting reason is good. You are not listening to me. No you are not listening to me etc.
There is almost always more than one thing to consider when diagnosing AKA building a 100% Coverage Problem Space Framework. A key strategy I have to consider multiple variables in a considered fashion to thread them together into a ‘100% Coverage Problem Space Framework’.
Then often you can go from negative sum debate to positive sum discourse.
Yes, I think reason 1 is important.
But I also think that reason 2 is important.
When we calibrate these reasons against the ‘100% Coverage Problem Space Framework’ we see there are four major variables to consider, and that both reason 1 and reason 2 are two of the four. Bolting them together through the ‘100% Coverage Problem Space Framework’ it appears the best tradeoff is to move forward with this recommendation.
Example: Using a ‘100% Coverage Problem Space Framework’ for building education resources
This is an oversimplification but I hope it helps.
Right now I think one of the first steps for building a product is figuring out and agreeing on a “100% Coverage Problem Space Framework = 1. Framework Destination + 2. Optimal starting point + 3. Systematic sequential path between the startpoint and Framework Destination”
For a Year 7 Secondary product, building a “100% Coverage Problem Space Framework” might take 100s to 1000s of hours.
Doing this means you can see the big picture of the problem space you are trying to solve. Not just focus on one piece of the picture and not know that you are doing this.
We are not collecting data points, we are building knowledge maps. I find a core part of building knowledge maps is creating and updating a ‘100% Coverage Problem Space Framework’.
Then you can calibrate the tradeoffs for an ‘Externally Supported Recommendation’ against the ‘100% Coverage Problem Space Framework’.
Let’s say you want to change the question recipe for a component of a secondary education resource:
Mini example 1:
Opinion: I think that my proposed change reflects part of the curriculum AKA not referencing the proposal vs the ‘100% Coverage Problem Space Framework’ and calibrating whether on balance your new proposal is highly likely to be a better outcome than the existing proposal.
Externally supported recommendation: changing the question recipe here is highly likely to increase the chance of students getting to the Framework Destination over the existing proposal.
Mini example 2:
Opinion: I think this change will improve the UX. AKA only considering one piece of the picture. AKA not referencing proposal vs the ‘100% Coverage Problem Space Framework’ and calibrating whether on balance your new proposal is highly likely to be a better outcome than the existing proposal.
Externally supported recommendation: while changing the recipe here would likely be a UX win it means we need to remove a component of our systematic sequential path which on balance likely means a lower portion of students get to the Framework Destination. So on balance it appears the proposal is a net loss. AKA considering the full picture.
Mini example 3:
Let’s say this is your ‘100% Coverage Problem Space Framework for Year 7 Maths’.
Optional Starting Point:
Scaffolded abstract maths
Systematic sequential path
Abstract maths
Introductory worded maths
Framework destination
High order (reasoning) worded maths
High order (reasoning) abstract maths
How do we define what ‘High order (reasoning) worded maths’ is?
Technically ‘reasoning’ can be almost anything.
A framework:
No variation = boring and rote learning
Optimal amount of variation = conceptual understanding
Too much variation = no learning as too much variation
So normally we need to define multiple types of reasoning, usually deduced from relevant tests (aka genomed) and then we need to figure out how many of these types of reasoning we can teach well.
The variation for reasoning isn’t ‘random’, it’s highly considered.
You can then calibrate the tradeoffs for different question variation proposals against the ‘100% Coverage Problem Space Framework’.
This may only make sense for people inside Edrolo.
If you only take away one thing
Either everything is equally important or everything isn’t equally important.
Clearly everything isn’t equally important.
Building a ‘100% Coverage Problem Space Framework’ is a core strategy to see the full picture, not accidentally focus on a piece thinking it is the full picture. AKA diagnose well.
Building a ‘100% Coverage Problem Space Framework’ is a core strategy to calibrate multiple variables at once.
Building a ‘100% Coverage Problem Space Framework’ is a core strategy to have people feel like they are heard and see the logic behind how they are contributing to a positive sum levelling up of a solution. Best. Sentence. Ever!