What do predicting the next US president and improving on-boarding of your product have in common?
As I write this, the US presidential election is less than a month away. The unique and arcane electoral college system used in this election, as opposed to popular vote, is not widely understood, especially outside the US.
To put it simply: it is a winner-takes-all system, but at state level. California has 55 electors out of total 538 for the entire nation, and Joe Biden will get all 55 electors whether he gets 1 vote more than Trump in California or gets all of the votes. This system has led to George W Bush and Donald Trump being elected President, despite more people voting for Al Gore and Hillary Clinton in 2000 and 2016 respectively!
One manifestation of this system is that there is larger focus on swing states i.e. states which might go either way — Democratic or Republican. Some states are decidedly Democratic (referred to as blue states — California, New York, etc.) and some are always Republican (red states — Texas, Mississippi). Hence, the fate of the election is often decided by the swing states (e.g. Florida) which might vote for either of the two parties.
So, to get an idea about the outcome of the US elections with limited resources, you are better off surveying all people of Florida rather than all people of the US, right? And if you can’t survey all people of Florida, you can survey, say, any 10% of the eligible voters in Florida to predict the results of the election, right?
We’ll come back to this.
On-boarding of new users as customers (aka Activation) is one of the key problems in Growth. Often activation % is the metric in the growth funnel you can influence most significantly. (Retention is representative of your product-market fit and becomes difficult to influence without significantly altering the value your product provides, or changing the market.)
Irrespective of domain, activation is about showcasing the value that the product is supposed to deliver as soon as possible, and in as friction-less a way as one can. A SaaS product will tell you its core features and ask you to sign up now for a free trial, no credit card required. A product like TikTok will start auto-playing videos without requiring a user to even sign up. A product like Twitter will try to get you to follow the ones with most followers as part of the on-boarding so that you experience the core value of Twitter — to see what’s happening around.
Now the value of the product is what the market perceives the product to be delivering, even if they can’t put a finger to it. So, in quest to improve activation %, you will:
- Ask all, or a sample of, activated users what is the value they find in the product
- Ask all, or a sample of, dropped-off users why did they not find value in the product
Right?
All voters in swing states are not swing voters, just that a higher % of them is undecided than in blue or red states. What is more pertinent to a poll seeking to predict the outcome of an election is not all the voters (or a random sample of voters) in swing states, but all or a random sample of the swing voters.
Similarly, not all the visitors to your product will become your customers. In fact, a majority of them won’t. So, while you can survey all the dropped-off users to collect data, while making a decision to change your on-boarding, it is important to not treat every data point equally.
Segment the dropped-off users into most likely to convert, moderately likely, and least likely, and then look at the survey data: you might get very different answers on how the value proposition of your product can be improved.
How to do this segmentation for a dropped-off user? Either based on the limited attributes of the user (and find ones similar to converted users), or based on their behavioral data on your product.
Putting it all together in an example, an e-commerce site might be converting 5% of its visitors into transactors but the rest 95% are not alike. So, if you have a WebEngage-like product on your website and have surveyed the visitors, you might be tempted to categorize the suggestions and implement the most common one. But wait, not so fast. Quantify the suggestions by the segment.
What are the ones who browsed a lot of product pages, used features like wish-list, login, etc., saying (most likely to convert — perhaps top 5%-10% of the 95%); what are the ones who viewed category pages and a few product pages saying (moderately likely to convert — perhaps the next 25%-30%); and what are the ones who bounced off, after a few pages, saying, if they are bothering to. You might come to a very different conclusion.
