The Middle Manager’s Guide to Metrics (1): Why your boss wants metrics
This article is part of my series “The Middle Manager’s Guide” with practical tips and lessons learned in my engineering leadership roles to help you thrive as a current or aspiring senior leader. It was co-written with Maggie Litton, Director of Engineering.
Tl;dr & Executive summary
Metrics are crucial, but require careful selection and management. Understanding what leaders at different levels need metrics for will help you balance those different needs:
The Critical Difference between Metrics 'For' vs 'About' Teams: Metrics "for" a team are used for local optimizations within the team. Metrics "about" a team are for external parties and often aggregated for insights at higher organizational levels.
Common Challenges: Metrics can cause tension when their purpose is unclear, or when they don't align with the objectives of different organizational levels. There is often resistance to new metrics systems, and metrics can be misused if not carefully managed.
Different Roles, Different Metrics Needs: Senior leaders focus on corporate metrics like ROI and NPS; they use metrics to obtain organizational visibility, and in one-off decisions. Middle managers are concerned with both higher-level metrics and team-specific metrics, creating a tension in balancing needs. Line managers focus primarily on team-level metrics to aid daily operations and decision-making.
Has your boss told you to come up with metrics for your team(s)? Or, are you getting questions about whether your teams are achieving their goals, and if they’re as efficient as they can be? Inevitably, all engineering leaders have to answer questions like these at some point.
Engineering metrics have been a hot topic in tech industry circles for a few years. But metrics are not just numbers; they're an embodiment of your team's objectives, a communication tool with upper management, and a lens through which to view the health of your organization. However, many leaders struggle with implementing and utilizing engineering metrics successfully.
We’re here to help you make metrics work with a step-by-step guide, giving engineering leaders a framework to define, implement, and leverage metrics effectively across various levels of the organization.:
We show what your boss and direct reports need regarding metrics. Like with performance reviews, people at different levels in your company have different goals and incentives. By understanding the different roles that metrics play at various organizational levels, you're better positioned to use them as a powerful tool for improvement, alignment, and accountability. Understanding and navigating those will be super helpful for…
Implementing metrics: The metrics we’ve found useful and how to make them work with your boss and teams.
Defining “metrics”
Metrics are often frowned upon by engineering teams; the reality is that they’re an essential tool for running a business. When I talk about metrics in this series, I refer to any quantifiable measures to track the performance of a business:
Engineering leaders typically use metrics to get insight into and improve the software development lifecycle. Examples include DORA, SPACE, or agile delivery metrics like cycle time.
Higher-level engineering leaders usually also deal more frequently with typical corporate metrics like sales revenue, MRR (monthly recurring revenue), NPS (Net Promoter Score), ROI (return on investment), or conversion rates.
The difference between “metrics for” and “metrics about” matters
One of the most significant sources of friction in an organization is when it’s unclear what purpose metrics are serving, especially if they’re “for” or “about” a group such as a team:
Metrics for a team: These metrics are primarily for the team to use together, e.g., to help them improve their work or make decisions. The main goal of the metric is to support local optimizations and learnings for the team.
Metrics about a team: These metrics show how the team has performed to other interested parties like executives. They’re often used in the aggregate to provide insights for decisions at a higher level and to speak to the organization’s performance. When your boss asks you to provide metrics, they are likely referring to metrics about your group. A word of caution: As a rule, I always share with people what metrics about their work I’m gathering and how I use them. Whenever you gather data about people, you owe it to them to at least let them access it - and may also be legally required to do so.
Many organizations don’t realize this difference, resulting in tension:
Metrics for teams aren’t always useful in aggregate. Your team may be very invested in improving developer satisfaction, while your boss is much more concerned about the ROI of engineering investments.
Metrics about teams are most meaningful at a higher level. Some metrics don't make sense at the team level, but start to become useful once you get a large enough sample size. That makes it hard to get people at the team level to buy into them. For example, the leadership team in an organization I worked with recently wanted to understand better if they’re investing in the right areas as a department. One metric they use for this purpose is the cumulative flowchart metric, which highlights investment distribution between features, technical debt, maintenance, defects, and escalations. They expected that not all teams would always adhere to these investments, e.g., because a platform team had a bigger maintenance project planned, while a feature team would spend more time on feature work. The engineering teams were aware of this metric and had access to the data about their team but weren’t using it (and some also felt it was a useless metric, understandably).
Especially as a middle manager, you will likely be working with both: The line managers reporting to you will use more metrics for their teams, while your boss will expect you to report on metrics about the teams.
The more overlap you have between metrics for and about groups, the more likely people are to embrace them - after all, making metrics useful is key.
Whenever you’re using metrics, be clear on who the metrics are for or about.
Why your VP/CTO/Head of Engineering asks for metrics
In a senior engineering leadership or executive role (I’ll use a VP role because that’s where I have the most direct experience), I’m accountable for everything about my engineering organization and need to optimize for the good of the whole organization. My job is to care about the value engineering delivers to the business, with an emphasis on value and business, not engineering.
As a VP, I’m often brought into companies to increase quality, improve efficiency, or ”professionalize” an organization and increase discipline to help them scale. I need to be able to confidently (!) speak to how the organization is doing. Therefore, I want to know how the organization as a whole is doing: If we’re doing the right things, how efficiently, and what risks need to be managed. Metrics help me get a good grasp on that: They help me manage, make decisions, and identify risks.
Senior engineering leaders use metrics to:
Inform the strategy for my organization. I also want to understand our history and status quo, in order to plan and navigate risks.
Know whether we’re on track to reach our goals and within budget, and course-correct if needed.
Own visibility across the organization. I need to know what’s going on and be able to speak to it. Especially in growing organizations, it’s common to get super specific questions from other executives, especially about critical projects on our roadmap or important customers.
Make one-off decisions, e.g., about organizational structure and investments.
Be accountable to others, provide visibility, and answer stakeholder questions (e.g., Board of Directors, Executive team, Sales teams). Many or even most of my stakeholders are outside of engineering and in business functions, and I need to be able to speak to their concerns - including sharing metrics with them that they find meaningful.
Know how the organization is spending money report on returns on these investments (ROI), or know how specific investments are paying off (e.g., hiring, new tech).
As a VP, I don’t (and shouldn’t!) care about metrics at the individual level. Even metrics at the team level don’t usually matter to me, with some exceptions: When a team is really off track or standing out for under- or overperformance, or there are questions about efficiency, I may review metrics at the team level. However, my first go-to person for questions about specific teams is usually their manager.
Therefore, as a VP, in order to achieve these goals I need to take steps that not everyone in the organization is super stoked about - and I’m (read: I need to be) okay with that. Some examples from my work history:
Introducing metrics: Engineers and managers often have questions and concerns when I introduce metrics to their organization, ranging from “why do we even need this?”, to “what does this mean for my daily work?”, to “what guardrails are we putting in place to avoid metrics being misused?”. Many engineers had poor experiences with metrics, from “lots of dashboards that took work to maintain, but that no one used”, to “metrics being weaponized against individuals”.
Introducing JIRA or other tooling: I know many engineers hate it with a fiery passion; I’m not a big fan of it either (it’s not my hobby), but it helps me get the job done.
While my role is ultimately to achieve my goals, I still want to address these concerns and questions, so I
Lead with transparency and context. I share a lot of the points I’m sharing with you here.
Hear feedback and understand where concerns are coming from, and what ideas people have to address them.
Work with my teams to address concerns. Metrics are best when they’re useful for everyone.
Middle managers & metrics
As a middle manager (think: Engineering Director, Senior Engineering Manager, and other roles managing managers), I see a broader swath than line managers. My concern is an entire sub-organization, domain, or at least team-to-team comparisons. I’m more concerned with the greater good and likely to accept global solutions and standardization, e.g., in process or tooling, than the line managers reporting to me. I know more than line managers and have more authority and power than they do, but still less than senior leaders and executives. Usually, I have more insight into the relationship between company goals like revenue and how my organization is working. At the same time, I’m concerned about how senior leaders handle metrics: Their approaches to this topic may undermine trust from the engineering teams and individuals at lower levels of the organization.
Therefore, I experience the tension between higher- and lower-level needs:
My boss (a senior engineering leader, e.g., VP) with purview across our entire organization, and
The line managers and teams reporting to me.
How do I balance these different needs? I want to include my teams in coming up with metrics. I also know that they’ll likely care most about metrics that are for them, which aren’t all going to matter to my boss: For one of my current teams, “uninterrupted focus time for low-level engineers” is a crucial metric because they’re trying to improve their work breakdown; however, this metric isn’t relevant to my boss’ current concerns.
The metrics that I ask my teams to report on and the metrics I present to my boss signal what I care about. In both directions, I want to use metrics each group cares about and finds useful, knowing that the specifics of what those are will likely be different.
Middle managers use metrics to:
Know and show that my organization is effective, efficient, professional, and well-run. Have visibility into my part of the organization and how my teams are working:
Is our team structure right?
Do teams have good interfaces with each other and other functions?
How do we compare with other domains, departments, or other companies?
Demonstrate that I’m focussed on our business goals and understand them
Balance the different contexts and needs of the people in higher and lower-level parts of the organization. Meet my boss’s expectations while having my direct reports on board and minimizing the emotional labor (e.g., contextualizing, explaining) with them.
As part of this balancing work, negotiate complicated tooling questions and pragmatic solutions, such as senior leaders’ desire for visibility and standardization, which needs to be balanced with creating too many (perceived) hurdles to get work done (“yet another mandatory field in JIRA?!”)
Collaborate with middle managers in other functional areas. People in departments like sales or customer success have very different incentives than engineering and often don’t understand engineering jargon. The metrics that I use need to be meaningful and understandable to them. My counterparts in customer support are likely more swayed by a metric like the number of incidents per month than engineering-centric metrics like cycle time or test coverage.
Line Managers & metrics:
As a line manager (think: Engineering Manager, Team Lead, Delivery Lead), I use metrics primarily to give my team and the individuals on it agency and purpose. Metrics help me and my team understand our domain and how we work, and we use this information to make decisions and change.
My job is to focus on execution and tactics. Through this work, I have the closest and easiest access to the daily experience of individual contributors, i.e., the engineers on my team. I also have more exposure to anecdotal evidence about how things work (we’ll talk about why this is useful in part two of this series).
My visibility into aggregates and trends is usually low. Therefore, it’s often easier to focus on local optimizations like improvements on my team, instead of spending lots of energy across teams. This limited purview can also make it hard to map my team’s work to the larger company strategy, or invest in large automation and telemetry projects if it’s unclear what direct value they provide to my team.
Introducing and using metrics always makes me and my team question how we will be perceived at higher levels of our organization based on these metrics, and what conclusions others will draw. Will metrics be misused, e.g., to label teams or judge individuals’ performance?
Line manager’s use metrics to:
Satisfy my boss and business needs
Minimize complaints and concerns from my team
Convey that I know what I’m talking about, understand the business, and have a good sense of what’s going on in my team
Minimize the measurement effort, as me or my team are most likely the ones who have to do the data entry and any manual collation
___
At all levels, metrics help leaders get the visibility we need to (1) do our job confidently, and (2) demonstrate to others that we’re doing your job.
Line managers, being much closer to engineers, usually understand their challenges and focus on local optimizations. Senior leaders and executives usually rely on aggregate data to make decisions. Middle managers have to balance these differences in contexts, purviews, and goals, which lead to tension in most organizations.
More in this series:
The Role of Metrics in Crafting Engineering Culture: How metrics shape behavior and create culture.
How to choose and use metrics: Learn how to select the right metrics for your organization, implement them effectively, and go from mere numbers to actionable insights, to create a culture of data-driven decision-making.
