Baseado em um post da Stephanie Leue, resolvi procurar mais sobre o que é esse TARS Framework…
Evaluate feature performance with TARS
Overview
PMs need to evaluate feature performance to get an accurate read on whether or not a feature is adding value to a product. But existing approaches like using product KPIs or focusing on adoption each introduce blind spots.
Instead,
PMs should leverage a feature-specific framework like TARS to understand feature performance. TARS captures how your feature is performing across four key data points:
- The target population for a feature
- The share of that target population who adopt a feature
- The share of those adopted users who are retained
- and the share of those retained users who are satisfied.
For low-frequency products like Turbotax and Carvana, you can skip over retention and go straight to evaluating satisfaction.
Make feature iteration decisions with TARS
The key question we want to answer is, “Does creating user value also create value for the business as a whole?”
There are two benefits to evaluating business value metrics.
- First, you can show why your feature is valuable not just to users, but also to the business. While your product leadership will appreciate user value, it’s even more relevant to be able to show how your feature contributes to the key metrics leadership is responsible for.
- Second, it helps you make more thoughtful iteration decisions. If you identify opportunities to improve user value through optimizations, but you realize your feature has already met its business value goals, is it worth investing in optimizing your feature? Or is it a better use of resources to work on a new opportunity?
By thinking through these questions, you demonstrate your ability to think strategically about how your work fits into the business as a whole.
TARS: A product metric game changer
1. Target Audience
As product managers, we must understand the problem we are trying to solve and for whom.
The first stage of the TARS framework focuses on this by first encouraging us to quantify the target audience by asking, “What percentage of all my product’s users have this specific problem my feature aims to solve?”
2. Feature Adoption
The second step in the TARS framework is to evaluate how well we are acquiring our target audience.
Once
we’ve pinpointed the percentage of our audience that has the problem
our feature aims to solve, the next step is to determine how many of
them actually use our feature to solve that problem.
3. Feature Retention
Retention is a crucial indicator of whether our feature solves the problem it was designed for and achieves product-market fit. When evaluating retention, we must ask ourselves, “Of all the unique visitors who meaningfully adopted our feature, how many came back to use it again?”
4. Satisfaction
As a product manager, it can be disheartening to realise that sometimes, despite all our best efforts, we’re actually making things worse. And it’s not always obvious — after all if our adoption and retention metrics look good, shouldn’t that mean we’re doing something right? But as I learned the hard way, it’s not always that simple.
You see, sometimes users don’t have a choice — they’re stuck using a product that drives them crazy, whether it’s because their bank has a great interest rate, their family only uses Facebook messenger, or their company makes them use Workday. These are what we like to call “Hidden Detractors” — features that people use but that are frustrating experiences. And trust me, these Hidden Detractors can come back to bite us if we’re not careful.
Hidden fifth step: Pulling all the metrics together
Finally, the piece de resistance, the cherry on top, the icing on the cake — pulling all the TARS metrics together into a funnel chart.
This visualisation is like a roadmap to product success, showing you exactly where you’re hitting the mark and where you’re falling short. And let me tell you, it’s a lot more effective than explaining it all with a bunch of numbers and jargon.
Using TARS for feature strategy
As a product manager, I often find myself in a pickle when getting my teams to row in the same direction. When planning a quarter, I want healthy debate and engagement from my engineers, designers, data scientists, and product leaders about the bets we want to take. But all too often, we set lofty goals like “increase revenue by 12% in 8 months” and hope for the best. This approach can lead to short-sighted actions prioritising short-term performance over long-term learning.
Thankfully, TARS is an excellent tool for avoiding such pitfalls.
S/T Score Calculation. Inspired by Reforge. Illustration by author.
See, TARS is more than a collection of metrics; we can divide the % of satisfied users by the % of target users for any given feature to create something called an S/T score. For example, if we had a TARS set up for a feature with a target audience of 80%, and then 50% of the target audience adopted, or 40% of the total, and then 70% of adopted users retained, which is 28% of the total, and then 90% of retained users satisfied, the S/T score for this feature would be 25% divided by 80%, which is 31%.
By itself, this S/T is not very exciting. But things get interesting when we apply the same calculation to all our other significant features to create a feature matrix comparison chart.
2x2 matric based on S/T Score. Inspired by Reforge. Illustration by author.
This chart plots our S/T scores on the y-axis and the feature’s strategic importance within our product portfolio on the x-axis. The more strategically important, the more it will be plotted to the right. This allows us to evaluate features against four criteria: over-performing features, core features, project features, and liability features.
Effectively we are plotting everything out on a 2x2 matrix from which we can evaluate features against one of four criteria: over-performing features, core features, project features and liability features.
Each quadrant has an exciting story to tell, and I’ve discovered that just sitting down with a team and looking at this chart is enough to stimulate some very spicy conversations about what’s crucial, what’s not and “why did we spend so much time on a feature that 2% of users use?”
But from a product manager’s perspective, the feature area we should be most interested in is liabilities. Liability features, in particular, require a lot of attention and effort to move them into a more helpful category. By identifying these features, we can focus our efforts on the areas that will significantly impact product growth.
The feature matrix comparison chart is an excellent example of how TARS forces us to take a step back and view our product from a three-dimensional perspective. It requires a high level of maturity from a team to acknowledge and take ownership of what is important and worth working on. But by implementing TARS, we can make informed decisions about what to prioritise and develop, leading to a winning product strategy.
The Dark Side of Feature Metrics: How Over-Reliance Harms
During the colonial occupation of India, the British were getting really concerned about the number of venomous cobras in Deli. To solve the problem, the British government offered a bounty for every dead cobra brought to them.
Initially, this was successful, with many snakes killed for the reward.Eventually, however, some enterprising people decided to breed cobras to collect the bounty. When the government found out, they scrapped the reward program, and the cobra breeders set their now-worthless snakes free, making the problem even worse.
It’s important to remember the saying, “What’s measured gets managed.” We must not be limited by the views of our dashboards and always strive to check if we have the proper perspective on the problems we aim to solve. When something goes right, we must ask ourselves why, and when something goes wrong, we must ask ourselves why.