These are the 4 steps of the decision making framework for your project:
- Identify key metrics
- Identify factors for respective key metrics
- Identify levers for those factors
- Identify leading indicators
Steps 1 & 2 help you to identify the problem (or opportunity area).
Step 3 helps you in identifying the solution(s) once problem has been identified.
Step 4 helps you track the progress once solution has been implemented.
Let’s go through the steps one by one.
1. Identifying Key metrics
- Do not begin the setup by making an exhaustive list of all possible metrics to track: Lists are not frameworks.
- Starting point should be to identify your key metrics.
- Generally the key metrics are foundational to the business: Profit, Growth rate.
- Key metrics generally come in pairs (vanity & sanity; main & check).
- My current model is to look for volume metric & health metric.
- For example, a marketplace startup wants to grow its revenue. My key metrics for them would be: number (#) of users (as volume metric) & revenue ($) per user (as health metric).
- Similarly, if it’s a free platform (news app, social media app, etc.), key metrics would be: # of users, & transactions per user. (Transaction as defined by the platform as the key action they want to maximise: opens, reads, messages, plays, etc.)
2. Identifying factors for the key metrics
- For each key metric, write down its formula in terms of mutually independent factors. Then write the formula for the factor. Follow this recursive process for 3–4 levels.
- For # of users, the formula might be:
# of new users + # of returning users
- Formula for both these factors might be:
# of new users = new users from channel 1 + new users from channel 2 + …
# of returning users = new users last week * week-1 retention + …
- For $ per user, the formula might be:
Transactions per user * Average transaction value
- Formula for both these factors might be:
Transactions per user = Visits per user * Conversion %
Average transaction value = Average basket size * Average selling price
- At the end of this step you will not have a list, but a hierarchy of metrics: key metrics at level-1, their factors at level-2, their factors at level-3, and so on.
# of users: level 1
# of new users: level 2
# of new users from channel 1 (e.g. Search ads): level 3
# of new users from the main Search campaign: level 4
Similarly,
# of returning users: level 1
Retention % of previous cohort: level 2, and so on
- You can visualize this metric hierarchy as org-level abstraction: CEO would regularly check number of users but maybe no further; Marketing head might extend to regularly checking channel metrics; Marketing manager might be going one level deeper into campaign reports; marketing executive might even know ad level numbers.
3. Identify levers for the factors
- The hierarchy you have at the end of step-2 is good for identifying problems or opportunity areas:
- If level-1 metric is increasing or decreasing, you can look at the factors.
- You identify the factors going up or down, and then look at their respective factors.
- At some point in this recursive process, you zero in on the reason driving the change.
- However, the point of building this hierarchy, and tracking the metrics in it, is to have the ability to make decisions that can influence these numbers.
- For this, you need to put levers against the lowest-level leaves/nodes in the metric hierarchy tree.
- For example, Conversion % is your level-3 metric that has been zeroed in as the root cause of a dip. You take funnel approach and break it further into level-4 metrics:
App open to category page % * Category page to product page % * Product page to cart page % * Cart page to purchase %
- Once, you have put the numbers against all level-4 metrics, you are still left with the question: how do I positively influence them to improve Conversion %?
- For this, you have to use domain knowledge and business-specific context to identify the levers driving the metric. This will be a function and not a formula; but you still have to identify the core levers than a long list of all possible reasons.
- For example, you might identify page load speed, product description length, number of product ratings, etc. as the driving levers, for product-to-cart % metric.
4. Identify leading indicators for the key metrics & levers
- We want to drive the level-1 key metric but the change starts at the lowest level in the hierarchy of metrics.
- Increasing revenue can be an organisational goal but it’s a broad goal for a project.
- An ideal project would be for a lever for level-3 or level-4 metric e.g. increase avg. number of available product sizes on product pages; or increase relevance of search results.
- If your hypothesis for metric-to-lever mapping is correct, the improvement in the lever would show up in the metric e.g. add to cart %, click on search result % respectively.
- If not, you have to build new hypotheses for which levers would drive the metrics, and then repeat the process.
- Key metrics are lagging indicators.
- That is to say, for example, # of your users on your platform today are reflective of efforts in the past, and are hard to influence in real-time.
- So, in order to gauge if the impact of projects you have taken on to influence the levers will percolate up the metric tree in the future or not (i.e. level-4 metric will improve, which will lead to level-3 improving, and so on to the top), you have to identify the early/leading indicators of the key metrics.
