Measurement-Driven Product Management
http://www.pragmaticmarketing.com/publications/magazine/8/2/measurement-driven-product-management
If you are a vice president, director or team leader for a product management function, one of the biggest challenges you face today is how to demonstrate the team is making a significant contribution to top line or bottom line targets. If you can’t measure your team’s effectiveness, or if you are focused on the wrong metrics, your headcount and budget allocation could be at risk. By Mike Smart
their employees to measure the wrong things."
Craig Stull, Phil Myers and David Meerman Scott in Tuned In
In a rapidly changing business climate the allocation of resources and budgets are closely tied to a department’s ability to get short term results. If you are a vice president, director or team leader for a product management function, one of the biggest challenges you face today is how to demonstrate the team is making a significant contribution to top line or bottom line targets. The pressure on budgets and resources from executives has forced a tighter link between current investments and near term results.
If you can’t measure your team’s effectiveness, or if you are focused on the wrong metrics, your headcount and budget allocation could be at risk.
In the 2008 best-selling book, Tuned-In: Uncover the Opportunities That Lead to Business Breakthroughs by Craig Stull, Phil Myers and David Meerman Scott; the authors point out that many companies force employees to measure and track the wrong thing which leads to out of synch or “tuned-out” behaviors.
In this economy, the best way a product management team can establish its value to the corporation is by using a reliable set of outcome-oriented measurements that demonstrate both performance of the product(s) and effectiveness of the role. The stakes are high! A product management function that demonstrates a direct contribution to the company’s income statement and uses objective measurements to gauge their effectiveness and value to the company will have more influence shaping the product strategy and the roadmap.
The fact there is increased emphasis on measurements, metrics and alignment with corporate goals during this business cycle is not a surprise. What is intriguing is the number of product management leaders acknowledging this focus will continue after growth returns to the industry. This is in part because CEO’s need better risk mitigation and are demanding more predictability in all aspects of the business.
At a recent ProductCamp industry event, a session on “Metrics-driven product management” was filled with product management team leaders, directors and VP’s. All of the attendees expressed serious concern about the need to establish better linkages between the activities of product management with broader company results such as revenue, product margin and profits. Very few challenged the sole use of these conventional corporate measurements as the gauge for product management effectiveness.
Success in building high performance teams begins with using measurements that give visibility about the overall effectiveness and efficiency of the team. The key is finding the right measurements.
What are the best measurements to monitor the effectiveness of product management teams? What are the best practices measuring the health of products? How to demonstrate value and alignment with larger corporate goals?
The Balanced Scorecard by Robert Kaplan and David Norton
Measurements and metrics
The use of measurements and metrics seems like a straightforward distinction but deserves some discussion. These terms are often used interchangeably, yet have very different applications. A more formal definition of the terms “measurement” and “metric” gives us a common language to move forward.
Metric: A measuring system that quantifies a trend, dynamic, or characteristic. Metrics encourage objectivity. They make it possible to compare; they facilitate understanding. Think benchmarks statistics and predictive indicators.
Measurement: A way of monitoring and tracking the progress of strategic objectives. Measurements can be leading indicators of performance or lagging indicators. Common measurements such as product revenue, profits, product margin and product adoption rate are often referred to as key performance indicators or KPI’s.
But, what type of measurements to use and how can they be used to accurately express performance and ultimately give management the ability to predict results?
There are three types of measurement that are necessary to create reliable performance indicators; rearview, operational and activity-based. All key to developing an effective product management organization. But the degree to which a company adopts a measurement driven approach to drive the business depends on the company culture.
Rearview measurements
Pragmatic Marketing’s 2009-2010 Annual Product Management and Marketing Survey shows performance in classical financial measurements such as product revenue, product margin and profitability are most commonly used to determine the success of product management. These measurements are “rearview” because they are lagging indicators; meaning they cannot drive the performance of people, processes or products. Product management teams that focus solely on these measurements usually struggle to establish clear value to the company’s goals.
Why do so many companies rely on rearview measurements to assess the effectiveness of product management? Because those measurements are easy to assign and consistent with the high level focus of the executive.
These rearview measurements are the natural tools of top-down goal setting. It is common for CEO’s to push these MBO’s (management by objective) down to product management. There is value in using rearview measurements; primarily to identify historical profit leaks such as:
- Product revenue growth
- Profitability
- Cost of sales
- Product margin
Rearview or financial measurements are part of the common language for companies. Product Management will be ultimately judged by the financial success of the product, but there are consequences with putting too much emphasis on these measurements.
- We learn too late what is working well and what is not
- Focus on rearview measurements can drive product managers to the wrong behavior
- Product managers are held accountable for outcomes they have little or no control in achieving
A solid understanding of the key financial indicators for products is important but it’s not enough to ensure a successful product or an effective team.
Operational measurements
Operational measurements are the “measurements behind the measurements.” These are leading indicators for most organizations, because they can drive and create outstanding financial results. Ideally VP’s and directors of product management interpret the company strategy and align with key financial measurements by emphasizing relevant operational measurements. These can be the key ingredients of a high performing product management team. Clear and relevant operational measurements enable better focus on the team and increase the probability of product and market breakthroughs. If we understand the relationship between specific operational measurement and the financial measurement, they bring more predictability to the company’s financial results.
Establishing and using operational measurements to evaluate the effectiveness of product management and the health of the product is critical because it bridges the gap between company strategy and execution.
A company’s operational measurements are how high-level goals become grounded. They become “the vital few” for the product management team.
Relevant operational measurements include:
- Market sensing—knowledge of market problems among customers, evaluators and potentials
- Speed-to-market—putting the right product in the right market segment faster than a competitor
- Product adoption—driving the uptake rate in a market at a lower cost and shorter time than a competitor
- Product launch—improving the response rate from target buyers with fewer impressions
- Customer satisfaction—increasing the “willing to recommend” percentage of existing customers
A crucial step is to translate the rearview or financial measurements into the appropriate operational measurements. The leader of the product management team must be fluent in both the financial and operational measurement paradigms.
| Operational Drive | Financial Outcome |
| Speed-to-market | Increase product revenue |
| Product adoption | Increase product margin |
| Product launch | Lower cost of sales |
Above, we see that linking these measurements gives the product management team more context and better insight into their contribution to the company goal. It also gives product management leaders concrete and objective measurement to track interim results and key milestones.
As an example, speed-to-market is a metric that can be tracked in several ways. How fast did we reach the market with new product releases compared to last year? How many times did we beat the competition to market with comparable product releases? Where do we rank within our industry segment? Over time we learn what effect improving this measurement has on increasing revenue. Every team member should understand these relationships and how their projects and key activities affect the outcome of the operational drivers.
Activity-based measurements
The next step in developing predictable measurements is linking activities, performance drivers and final outcomes.
Activity-based measurements are the execution side of the measurements fulcrum. While operational measurements are leading indicators to financial results, activity-based measurements are the tactical tasks that lead to the desired operational outcome. Using these allows team leaders to identify the crucial activities that drive the desired outcomes. Activity-based measurements reduce ambiguity and establish accountability for all team members.
Activity-based measurements that a product manager should be held accountable for are:
- Onsite market interviews
- Assessing impact to customers for existing or future products
- Positioning to buyer personas
The table below illustrates how to close the gap between specific activities, operational drivers and financial results.
Activity-Based Targets | Operational Driver | Financial Outcome |
On-site interviews | Speed-to-market | Increase product revenue |
Assessing impact to customer | Product Adoption | Increase product margin |
Positioning to buyer personas | Product Launch | Lower cost of sales |
Properly conducted onsite interviews across the total addressable market enable discovery of un-met needs or market problems. Validation of these market problems through surveys demonstrates what percentage of the market has these problems and the value of solving them. The data from these activities gives product managers the facts to define the next release and a high degree of confidence that the proposed solution is the right product for the right target segment. This product development approach is fast and more efficient.
Establishing strong links between activity-based measurements and operational measurements may not be easy, but is critical in establishing the value of product management and to build credibility within the organization. The key is to develop reasonable correlation and monitor the accuracy of these relationships over time.
Here is an example of how a team could establish strong links between activity-based measurements and operational outcomes.
The annual operational goal for the product management team is to improve speed-to-market by beating competitor A to the market. What activity-based measurements must the team leader assign to each product manager to ensure this goal is achieved by the end of the year?
Consider these:
- Conduct Strength, Weakness, Opportunity and Threat (SWOT) analysis against Competitor A
- Complete customer impact assessment of Competitor A’s products vs. ours
- On-site customer visits to validate Competitor A’s perceived product gaps
- Complete Market Requirements Document (MRD) with Competitor A as key theme
- Deliver 100% of something ahead of Competitor A
One caution to the team leader—make sure the sum of these activities is greater than or equal to one or more of the team goals. This linking can provide the ability to benchmark and propose metrics that have predictive value.
The key to success with this approach is to continuously review data and trends to ensure the measurement remains relevant.
Best Practice Measurements
Share of hearts, minds and markets
Most product management leaders would benefit from the ability to rigorously assess existing projects and new opportunities, identifying potential risks. This approach would guide investment and allow VP’s and directors to evaluate outcomes and reinvest to maximize contribution. The enhanced visibility would enable product management leaders to defend the allocation of resources with measurements that are tightly linked to the company’s income statement.
As a leader of a product management team, what should you measure?
Initially two things: product performance and product management effectiveness.
These are different but equally critical dimensions of the product management process. Team leaders must monitor people, process and market success for the product lifecycle. To be successful, all three must be continually assessed.
Must be relevant
The most important outcome from Product Management are financial results, but product management leaders must translate the financial results into performance drivers that lead to the income statement outcome. Until a clear linkage between actions and outcomes is established the measurements are theoretical.
When it comes to setting targets for individual product managers there are best practices to follow.
Look for connections between operational measurements and activity-based measurements that may not seem obvious at first. Question your current assumptions about the business and what really drives product performance.
Require transparency
Initiating a performance measurement program is a commitment to changing the way a team operates. The biggest change is a commitment to complete transparency.
In some companies, Product Management has accountability for the operational goals—each of these are translated into product specific objectives and measurements. The product management leaders must defend key measurements and performance at the product line level.
Must impact financial results
In a down economy, CEO’s are pressured to emphasize near term results and near term execution that impact the income statement. Product management leaders must be aware of potential shifting priorities and if your key measurements are trending badly, early intervention is critical.
The team leader using this measurement system must have a dashboard view of products, projects, processes and people. This is the only way to focus on what matters most and to continuously refine key measurements. It also allows the team leader take action based on insights gained or emerging trends.
However, the proper use of measurements must go beyond driving short-term financial results. The real potential is in changing the way Product Management thinks about its role and value to the organization.
Ultimate goal
“Every metric, whether it is used…, to evaluate future strategies, or simply to take stock,
will affect actions and decisions.”
From “Metrics: You are what you measure”
European Management Journal
The long term benefit of Product Management becoming measurement-driven is higher team performance, improved predictability and increased credibility. The ultimate benefit is developing the ability to reliably create outstanding products and market breakthroughs.
Can Product Management operate with this high level of maturity, using a reliable measurements and metrics system with more predictable results in a company?
This “holy grail” of product management performance is doable, but often many cultural and process gaps must be addressed first. An organization fosters a measurement-driven culture by reinforcing other aspects of the process, such as tightly coupling rewards, recognition, compensation and promotion to attainment of operational results. Does yours?
Mike Smart is an independent consultant working with private equity firms to increase the value of their portfolio companies by implementing product management and product marketing best practices. He has a passion for identifying key metrics that drive company performance. Mike brings more than 25 years of industry experience in a variety of leadership roles. These include Senior Vice President of Operations, VP of Product Management & Development, VP of Product Management and VP of Sales.Most recently, Mike was an instructor and consultant with Pragmatic Marketing where he worked with medium and large enterprise software companies to teach and implement the Pragmatic Marketing framework. Contact Mike at msmart@egrss.com.
