Bulletproof Problem Solving

Introduction to Problem Solving in the 21st Century

• Core skill for the 21st century workforce: defining a problem, breaking it down to manageable parts and working towards a solution
• Definition: problem solving is decision makig when there is complexity and uncertainity in determining answers and consequences to make work to get good answers worth it
• Complex problem solving is now needed more than ever and should combine mental muscle with machine muscle
• Job growth is focused in areas where tasks are non routine and cognitive
• Seven step process of problem solving:
  • How do you define a problem in a precise way to meet the decision maker’s needs
  • How do you disaggregate the issues and develop hypotheses to be explored?
  • How do you prioritize what to do and what not to do?
  • How do you develop a workplan and assign analytical tasks?
  • How do you decide on the fact gathering and analysis to resolve issues while avoiding biases?
  • How do you go about synthesizing the findings to highlight insights?
  • How do you communicate them in a compelling way?
• Pitfalls with current problem solving approaches:
  • Weak problem statements: not specific, not clear about criteria and constrains, no indication of action that will occur if problem is solved, time frames or required accuracy
  • Asserting the answer: answers based on experience and analogy rather than actually testing if solution is good fit for problem at hand. Corrupted by avaliability bias, anchoring bias and confirmation bias
  • Failure to disaggregate problem: Need to break apart problem into components and see which components have the most impact on the problem
  • Neglecting team structure and norms: should have diversity in how you apporach a problem
  • Incomplete analytic tool set: you need the right tools to do well
  • Incomplete synthesis: make your findings matter to the audience
  • Problem-solving is not treated as an iterative process:

Chapter 1: Learn the Bulletproof Problem Solving Approach

• It is both a complete process and an interative cycle: complete in one time period and thn keep repeating the process
• Should have a coherent summary of understanding of problem and solution path at any point in the project, not just at the end (one-day answers)
• Step 1: Define the Problem:
  • Without proper context and boundaries, there’s a lot of room for error
  • Arrive at a problem definition agreedd upon by those involved in decision process
  • Should be specific, can clearly measure success, bounded by time frame and values of decision make, involves definitive action
• Step 2: Disaggregate Issues
  • Use logic trees to disassemble problems into parts for analysis driven by hypotheses
  • Try different cuts to see which ones yields the most insights
• Step 3: Prioritize and prune
  • Idenfity which branches of the tree have the biggest impact on the problem
  • Employ a matrix of sizie of impact of each lever and ability to move lever
• Step 4: Build a Workplan and Timetable
  • Assign team members to tasks with specific outputs and completion dates
• Step 5: Conduct critical analyses
  • Will need heuristics and complex problem solving techniques to do well
  • Use one-day answers to keep on track
• Step 6: Synthesize
  • Asssemble findings into logucal structure and convince that you’re solution is right
• Step 7: Prepare Communication:
  • Develop storyline from concluions to link problem to solution
• Logic trees are extremely useful tools: provide clear visual representation of problem, holistic, lead to clear hypotheses that can be tested using data
• Use mathematically complete and deductive logic trees for business problems, weighted factor analysis for decision making, and decision trees fro walking through complex choices

Chapter 2: Define the Problem

• Good problem statements: outcomes focused, specific and measureble, clearly time-bound, explcitly addresses decision-maker values and boundaries, structured to allow creativity, solved for highlest level possible
• Another way: SMART (specific, measureable, actionable, relevant, timely)
• Important to find the right scale and scope in problem definition
• Advantageous to allow flexibility for creative solutions
• Try to have as large of a scale as possible to account for individual differences that pop up (eg. instead of focusing on saving salmons in Canada, focus on North Pacific scale, so that decreased fishing in Canada not offset by increased fishing in Russia)
• Sharpen the problem at hand by using data
• You can use design thinking principles in conjunction with the 7 step method: add active empathy for user

Chapter 3: Problem Disaggregation and Prioritization

• Any problem of real consequence is too complicated to solve without breaking it down into individual logical parts to help us undertstand the drivers and causes of the situation
• Try to make at least 2-3 different disaggregations via logic trees to determine which n=one yields the most insight
• Types of logic trees:
  • Factor/lever/component: divide problems into components when you don”t know much about structure of problem
  • Inductive: build from end points to problem. Used when you don’t undertsnad the underlying structure/relationship between the components
  • Deductive: Break problem into smaller relationships and factors. Used especially when you have a clear idea of the problem structure
  • Hypothesis: know enough about the structure to make clear hypotheses to test data
  • Decision tree: uses if-then analysis
• Logic trees should be MECE: mutually exclusive and collectively exhaustive
  • Mutually exclusive: branches of trees don’t overlap
  • Collectively exhaustive: tree contains all elemenst of the problem, not just some of them
• Constantly iterate on your trees so that it gradually becomes more MECE
• Deductive logic trees:
  • Usually a tree where individual components mathematically complete and add up to the desired objective of the problem statement
  • Ex: Return on capital tree uses the formula return on capital = profit margin * capital turnovers. Useful in comparing firms
• Inductive logic trees:
  • Working from specific observations to general principles
  • More probabilistic becasue underlying structure is not known
• Prioritization:
  • Need to prune our trees so that we are focusing on the most important issues in our problem
  • Prioritization matrix: y-axis is low/high potential scale of impact, x-axis is low/high ability to influence (can we change this lever?)
  • If prioritization is hard, then consider making the problem a little more open (nit as many constraints)
• Cleaving frames: different frameworks of logic trees so that it makes analysis much easier
  • Business
    • Price/volume tree: helps to understand what is the nature of competition in the market
    • Principal/agent: used when there are contractor (agents) that do work on behalf of principals (investors/managers). Used for second-hand asset sales and insurance problems
    • Assets/options: every asset creates options for future strategic moves
    • Collaborate/compete: Based off game theory
  • Social
    • Regulate/incent: lawmakers often have to decide whethr to regulate or use incentives for people to adopt behaviors
    • Equality/liberty:
    • Mitigate/adapt: contrasts policy efforts to reduc harm from causal factor or adapt to factor
    • Supply/demand:
  • Since many different cleaves could lead to several important insights, make sure to test several
• Use teams to help you make insightful cleaves and logic trees\

Chapter 4: Build a Great Workplan and Team Process

• Workplanning and constant iterations are how you stay on critical path as defined in prioritization
• Best practice at Mckinsey: don’t do any analysis if no hypothesis, need to know WHY you are doing every single analysis, visualize the output, order analyses based on logic, put painstaking detail in who does what and the deadlines

• Avoid the tendency to jump straight into data without thinking what the output should be
• Order of analyses:
  • Knock-out analyses: analysis that will obviate later analyses, reducing work
  • Really important analyses
  • Nice-to-have analyses
• Knock-out analysis requires thorough prioritization. Use the 80/20 rule
• Chunky and lean workplans:
  • Short and highly specific workplans that last 2-3 weeks
  • Iterate from the plans after intial analyses conducted
  • Couple with Gantt charts to chart overall project progress
• One-day answer: beds down any analysis and observations. Can pursue promising pathways quicker. Typical method: *
  1. Status quo before anything begins
  2. Set of observations that captures tension/dynamic of problem
  3. Best idea/resolution to address right now
  • Crisp and concise

• What makes a good team?
  • Team Structure and Leadership:
    • Usually have an excellent lead that is a mix between a conductor and an air traffic controller
    • Relatively flat structure to ensure great brainstorming
  • Team Norms
    • Hypothesis-driven and end-product oriented
    • Porpoise frequently between data and hypothesis: if data disproves hypothesis, they halt and move to another analysis path
    • Look for breakthroughs rather than incremental improvements
  • Team Behaviors to Avoid Bias and Error
    • Have an extremely diverse team
    • Try multiple trees/cleaves
    • Question your hypotheses
    • Team brainstorming practices:
      • Obligation to dissent
      • Role playing: switch between different stakeholders to undertsna their point of view
      • Dialectic standard: every idea must have an antithesis and then synthesized
      • Perspective taking: look at other views of the problem and try to take that as your own to understand their point of view
      • Constructive confrontation: ask ‘what would you have to believe’ to understand what makes up a particular opinion
      • Team distributed voting: each person has 10 votes and people can vote where to prune the tree. This avoids sway from senior figures
      • Solicit outside views: talk to all stakeholders, not just experts
    • Explicit downside scenario modeling and pre-mortem analysis: based off Black Swan effect. Look at all unlikely failures
    • Good analytic techniques
    • Broaden data sources

• Agile team processes: based on the same 7 step model

Chapter 5: Conduct Analyses

• Good problem solvers have a toolkit at hand filled with heuristics and rules of thumb to understand direction and magnitude of relationships, which can be used for further prioritization
• Do not jump into building complex models without fully understanding whether these complex tools are even needed. This is why heuristics are needed
• Avoid torturing the facts to get what you want

• Can use 5 Ws and H to come to relatively rough-cut solutions: can put a few of these questions into a decision tree
• Root cause analysis: 5-Whys, Pareto Principle, Fishbone

Chapter 6: Big Guns of Analysis

• First ask yourself: have you adequately framed your problem and the hypothesis that you want to test such that it is obvious that you need big guns?
  • For most of the time, the last chapter’s tools will suffice
• If the problem is truly complex, you have to consider:
  • Is there data to support the analytical tool?
  • Which tool is right for this problem?
  • Is there user-friendly software? Can we outsource this?
• You need to be focused when using these big guns. That’s why you should use order-of-mag tools as listed earlier to understand in which general direction you should be investigating
• First do some basic exploratory analysis to see any patterns/trends in the data

• Bayesian statistics is particularly useful when your data is limited and possibility of experimentation is low
• Game theory can be useful to determine the best strategy in a complex game-like situation

Chapter 7: Synthesize Results and Tell a Great Story

• ”The quality of your synthesis will determine the quality of your decision-making. That’s why it pays to triangulate your views with people who you know synthesize well” - Ray Dalio
• Final synthesis: draw work from individual hypotheses and showcase in a visual manner
  • Best if you have one visual for each branch/split-off point
• Good team process is important, as you need the whole team to synthesize the project
• Create a pyramid-like structure when presenting your results
  • Top-level: governing problem statement. Use your latest one-day answer
  • Fill in supporting arguments to top-level answer

\

• Vary up the way you structure your arguments: use a combinaton of deductive and inductive construction
• If a client is particularly difficult or doesn’t want to hear your solution, conside r using a decision tree structure than the pyramid structure; provides an option for them to bail out on your recommendation
  • This reveals your answer rather than starting out with an answer. This helps build the argument in the client’s mind

Chapter 8: Problem Solving With Long Time Frames and High Uncertainty

• Uncertainty can be classified under 5 levels, with Level 1 being a reasonable prediction of the future and Level 5 being unexpected/unforeseen circumstances
  • Level 2: Alternative futures due to change with binary results. Uncertainty is resolved in a 5-10 year period
  • Level 3: Range of futures possible but which one is most likely isn’t clear. Resolved in 10-20 year period
  • Level 4: True ambiguity as you cannot concieve all potential outcomes
• Don’t be afraid of uncertainity! Having a useful mental model that is slightly smarter than others can help in the long runs
• Actions you can do to address uncertainty:
  • Buying information: gather more data and perform more analysis. Level 3 and Level 4 events may require some modelling. Likely to be high cost towards moving to perfect information
  • Hedges: make a reasonable cost move/investment to counter downside
  • Bet on multiple horses: account for as many different ways to make best use of future
  • Insurance
  • No regrets move: perform action becasue you are uncomfortable with uncertainty, rather play it safe
  • Big bets: you have a level of confidence in outcome not shared by anyone else and you act on it (think Big Short)
• Best to think probabilistically for long-term uncertainty