What Engineering Metrics Should I Use? A guide for Engineering Managers, Directors, and VPs
The question I get asked the most is, "As an engineering leader, what metrics should I use? Which ones are actually useful? Can you give examples?" I find this a complicated question to answer in a useful way without asking many context questions, but I also get it: It's helpful to hear what others have done! The exact metrics that you choose are an important, yet only small piece of a puzzle:
Metrics create incentives,
that create behaviors,
that create culture.
In my article series about metrics, you'll find several guides to help you think about metrics and understand the context that will help you choose the ones that are best for you and your team(s):
Understand what leaders at different levels of your organization need. Your executives, boss, and direct reports all have very different motivations and desired outcomes from using metrics.
Understand what's best for your organization and team(s). It doesn’t matter what framework you use, and whether you pick DORA, SPACE, DevEx, agile metrics, employee engagement, or some handcrafted metric that’s pulling data from two databases and a spreadsheet: Metrics only matter if they align with your organization’s goals and you’re actually using them to inform your behavior.
Understand the problem you're trying to solve. Metrics are a tool to understand and solve a problem; they’re not the solution.
Measure not just what you’re doing, but also how you’re doing it.
How you select and roll out metrics will go a long way to making this undertaking successful.
All that said, it’s time to let the cat out of the bag: There are metrics that I've found helpful in my work with various organizations from early-ish-stage startups, to scaleups and public companies with several thousand employees. Here they are, and how I came up with them.
___
Useful metrics for leaders at different levels
In this section, I'll share with you metrics that I found helpful when running a team, a set of teams, as well as entire organizations & departments. A few contextual notes:
These metrics are based on my experience as a leader working with various organizations, and my work as an organizational developer, executive, coach, and startup advisor; they are not reflective of a singular organization I worked with in the past or am currently working with. Your mileage will likely vary.
I'm categorizing them between efficiency- and effectiveness-focused metrics; businesses need to not just do the right thing (effectiveness), but also do it the right way, and avoid creating waste along the way (efficiency). I always recommend to measure some of both to avoid getting a skewed view.
The metrics I select are primarily influenced by two factors:
Why I use metrics as an engineering leader
Reason 1: Metrics create culture
Metrics are an important tool for creating the right incentives and a positive, productive culture ("culture" meaning "the behaviors that we reward and punish"). In my teams/organizations, metrics help foster
strong alignment ("we all know what we're doing"),
visibility ("everyone has insight"), and
accountability ("we all know how we're all doing, whether we're achieving our goals") as well as
continuous feedback, learning, and improvement ("we see the impact of our actions, learn from mistakes, and iterate from there").
The metrics that I choose to achieve this goal are focused on being metrics that everyone in my team/organization uses actively to help them improve over time ("metrics for a team").
Reason 2: Reporting
As a leader, it's my job to be able to speak to my organization's performance at all times; knowing how we're doing and knowing how confident I am in that assessment is a critical part of my job. This also includes reporting to others (my boss, product counterpart, and other stakeholders), and being accountable for my decisions.
In order to achieve this, I select metrics that help me identify patterns within and across my org, as well as help me compare to others (e.g. other teams/groups). I'll also choose metrics that help me feel confident that I actually know how we're doing. In contrast to metrics for my team to use actively themselves, these reporting-focused metrics are typically more "metrics about my team/organization”: My teams will be aware that I’m using them, but they’re less relevant to them.
Takeaway for you: No matter what level you're at, identify what metrics are for vs. about the people in your group. You will typically need a mix of both; this guide has more on how to handle this distinction well.
But not all metrics make sense for each organization at all times. So -
When should you be able to measure what, and how do you get there?
At least initially, many teams don’t have the tools and processes in place to measure a lot of their work: An early-stage startup may use a very simple project management tool as their software development process. And that’s okay! At this stage, anything more would probably add too much overhead and slow teams down. This is why, as always with this topic, context matters.
That said, your project management/issue tracking and software delivery tooling will have a drastic impact on your ability to measure anything, and migrating such tooling usually becomes a big headache very quickly. So here’s what I typically recommend for what engineering team size; all numbers are directional for growing organizations:
1-10 - Everyone works more or less on everything and the team size is small enough so that everyone still has visibility into what’s going on. Use a rudimentary project management tool.
~8-30 - You split into more and more teams and visibility decreases. During this process, I highly recommend adopting a robust project management tool like Jira, Asana, or Basecamp. Most of my experience has been with Jira; I know it’s not very popular, and I wouldn’t call using it a personal interest of mine either, but it’s beneficial: It’s a robust platform that integrates with many others.
~25-50 - Your organization is growing. Visibility across and between teams naturally decreases, while complexity increases. This means that your delivery process should become more robust. At this stage, I highly recommend standardizing some aspects of your issue and project management across all teams: Identify which aspects you and your business need to have visibility into, and how you’ll monitor them. In practice, this could mean making some fields in your project management tooling mandatory, such as:
Issue type - so you’re aware of how your teams are investing between new investments, maintenance, bugs, and technical debt ( → agile flow distribution metric)
Priority - at least for bugs and other escalations
Standardizing Kanban or Scrum board columns across teams - having the same columns will allow you to get a view where work gets typically stuck
Note: Resist the temptation of making too much information mandatory and standardizing too much. Teams having autonomy (within guidance) is an important part of scrum and especially agile work, and of setting your teams up for success. Consult your teams as you bring more process to them.
At later stages, there’s typically some rudimentary visibility system in place that can be build on; if not, it’s time to introduce one! But you may still have a concern:
What if people game metrics?
The concern that people may game metrics, i.e., change their behaviors based on metrics, only in order to improve the metrics, comes up a lot. My thoughts:
This is typically a negligible risk in a healthy team/organization where metrics are used as I recommend (balanced, transparently, and in collaboration with your teams, as opposed to a monitoring and staff discipline tool).
You may also sometimes deliberately choose metrics that you want people to game (e.g. rate of pull requests per developer, because you want everyone to ship work in smaller chunks).
And: If your organization genuinely needs to worry about people modifying the way they work drastically, only in order to appease management or comply with some metrics, you have much bigger problems than you probably realize. (Read any metrics-related thread on r/softwaredevs if you're unsure what I mean.)
Now let’s dive into metrics that my teams and I have found useful in the past:
Useful metrics for Engineering Managers, Team Leads, & Technical Leads or Staff Engineers
When I manage a team of engineers directly, I’m most interested in metrics that help me
Spotlight areas that my team cares about and improve together,
Identify patterns within my team, and
Convey to my manager how my team is doing.
As an engineering manager/team lead or technical leader on a team, you are in a unique position to improve your company's bottom-line: You and your team decide what the team works on every day. You see the small bits of disruptions, annoyances, etc. as they add up. At the same time, you have a good idea of how individuals on your team are doing. You understand if we're achieving our goals (we're effective) and how much waste we avoid (we're efficient) along the way; it will help if you measure at least some of both.
Metrics that my team cares about
Ultimately, teams want some answer to:
What are we getting done? What's slowing us down (or helping us!) in getting things done?
Are we getting better (as a team and as individuals)?
Teams also often ask for metrics that cover an area that’s a pain point for the team, such as developer experience and satisfaction, build time, amount of uninterrupted focus time, or time spent in interviews. I collaborate with my team to hear what’s important to us, and how we want to measure it - and how we use those metrics. We usually also use some of the metrics listed below together!
Metrics to identify patterns within my team & report about my team
As mentioned above, there are two aspects to reporting: First, your own confidence that you have a good sense of how your team is doing, and second, making sure your stakeholders and boss know. So what metrics should you regularly look at?
As an engineering manager, you should usually be able to speak to:
How does your team's work contribute to our higher-level goals?
How are you progressing towards these goals?
What are our risks? How are you handling these risks?
Overall, are we getting better?
And, with each of these: How confident are you in your answers to the questions above? And what gives you this (low or high) confidence?
I always want a set of metrics that cover my team, our systems, and the product. Useful metrics can be:
Team effectiveness measures ("are we doing the right things, well?")
Doing the right thing
Distribution of escalations, bugs, tech investments, new features (agile distribution flow chart)
Achieving our goals
Progress against our goals, like % of accomplishment of our quarterly goal
Are individuals achieving their goals? (This shouldn't be reported back to the team, but is important for your own visibility)
Delivering quality
MTTR, MTTI
Customer feedback
Availability, reliability
Employee experience and satisfaction
Employee engagement survey results
Team efficiency-focused metrics ("how much waste are we incurring?")
Visibility. Especially in earlier-stage or less mature teams, the question alone of "how much visibility do we actually have into our services and work?" is often really hard to answer. This includes:
Our services and any observability, monitoring efforts
Reliability and reliability issues
Dependencies with other teams
KPIs
Quantitative estimates of the effort to collect the data, especially if it’s high overhead
Getting things shipped
Some measure of throughput, like velocity, cycle time, or ticket numbers
Amount of uninterrupted focus time
Time spent in interviews
Interruptions. This depends a lot on maturity and process, but e.g.
Backlog health - usually qualitative, including: Are items organized? Is work actually being pulled from the backlog, or coming in from other places?
Predictability of work
Story points or another estimation measure
Planned vs. done work
Learning and improving
Onboarding time for new hires, e.g. "time to 10th, 20th, and 50th PR"
PR review time. This can be really useful - as an example: PRs for service A take much longer to get reviewed on average, than PRs for other services. This can point to various issues, like knowledge silo, need for knowledge-sharing, or a gap in people’s understanding.
Metrics trends overall
Metrics that I have not found useful at the team level, and why
It’s worth mentioning because some metrics are easy to cause harm with:
Lines of code (business value isn't just made from creating code)
Number of commits (not useful without any context)
Number of features delivered (incomparable data points, too many dependencies, and just releasing a feature doesn’t mean it will actually incur business value)
Lead time (only useful at the team level in organizations with a very, very strong product discipline and delivery process)
Story points, t-shirt sizes. These are classic metrics for teams: They’re not even metrics, but estimation tools. Please do not use estimation metrics to compare one team with another. They are absolutely meaningless and any attempt will backfire. The main and only useful reasons for tools like these are to help teams
get better at estimation,
provide a forcing function for breaking down work into smaller chunks, and
be a tool to have important but hard conversations, e.g. about how work is shaped, the requirement management process
Number of meetings or time spent in meetings, except when your meetings are purely a waste of everyone's time. Worth noting: At the team level, meetings are typically small enough that all team members should be able to actively contribute in making them useful meetings. If not everyone finds them useful, that’s often a fixable issue.
Individual-level metrics: Aside from voluntarily self-reported information (e.g. feedback and engagement surveys), I typically do not use quantitative metrics to determine performance at the individual level.
My job is to build high-performing teams and organizations that help our company achieve its goals, an inherent part of which is focusing on results; my job is not to monitor keystrokes.
Useful metrics for Directors or Middle Managers
Here are some that I’ve found helpful as a Senior Manager, Director of Engineering, or in similar middle management roles where I’m managing managers and a domain/set of teams:
Effectiveness measures
SLOs/SLAs with other internal or external groups: for applying security patches, for triaging customer support issues, number of false positive pages, etc. This offers a perspective on workload and collaboration, and helps us prioritize.
For assessing line manager/team lead/org effectiveness
Survey of self-reported perceived quality of team meetings
Average # of engineers per project (gets at WIP, amount of collaboration)
Self-reported level of satisfaction with career growth opportunities, feeling heard and taken seriously with feedback and concerns, and manager support
# of experiments conducted (not just in the growth engineering sense–think process changes as well)
Goal achievement
Planned to done ratio: Are we delivering what we committed to?
Efficiency metrics
When trying to reduce knowledge silos or the risk of depending on single individuals (“bus factor”):
Number of different contributors to a particular repository that previously had a single maintainer. If we’re reducing the knowledge silo, this number should go up.
Percentage of individual contributors on-call. Measures the distribution of on-call responsibilities.
Measure of toil/inefficiency
% of different types of work (innovation, support/remediation, overhead, unplanned urgent items, etc.)
When trying to break up a monolithic codebase into micro-services and discrete components, I wanted to test how well we’d mapped service ownership to our teams. I looked at a few different aspects:
Number of PRs requiring cross-team review. If we’d done a good job defining team charters, this number should get lower over time.
Customer support responsibilities. Tracking the number of tickets or incidents per team member measures the workload and efficiency of customer support. Average cycle time of support tickets is important too, as it assesses the responsiveness of teams to customer issues.
Number of simultaneous projects per team, or number of projects per a certain timebox roughly normalized for size (also often “work in progress”, WIP). This reflects the team’s workload and is an important signal for where we may be causing waste: Context-switching is expensive in software engineering, and too much work in parallel typically correlates with work not getting actually finished. High WIP is usually a smell.
Average age of open bugs of a certain severity, per team. This metric tends to be a bit tricky - the older teams get, the higher this number will often get. Eventually, you’ll likely have to focus on just the most important ones, often p0 and p1 (priority zero and one).
Useful metrics for Senior Engineering Leaders (CTO, VP, Head of Engineering,...)
As a senior engineering leader, my main job is to help my organization achieve our goals. In order to do so, I want to understand how my organization is doing, identify and manage risks, and convey it all to business stakeholders like my boss, peers, the executive team, and our board of directors.
I typically use a lot of the metrics that middle managers use as well, just mostly view them in aggregate across my entire organization. At my level, patterns and trends are major focal points for me, so with any data I review, those will be part of it.
I usually only dive into lower levels if there are great variances or unexpected data points. The organizational view is most important for me in the day-to-day, the metrics I find most useful answer the following questions:
Effectiveness measures
Are we achieving our goals? - E.g. OKR attainment, SMART goals, product and technical roadmap vs. reality
Are we doing the right things? - Goal alignment vertically (goals trickle down from the company vision and strategy to the team level) and horizontally (alignment with stakeholders and partners)
How and where are we investing?
Budget and how we distribute it
Agile flow distribution indicating how and where we’re investing in our delivery
Performance of technical systems over time
Employee feedback about investments and prioritization
What quality are we delivering in?
Ratio of opened to closed defects/bugs
Change failure rate (DORA)
Time to restore service (DORA)
Uptime, availability
MTTR
Service health
Customer satisfaction
How are our customers using what we built?
Sign-ups, customer adoption, customer retention
Conversion rates
Feature usage
Customer NPS and other feedback
How are we doing as a leadership team, including my direct reports and other managers in my organization?
Employee feedback
Onboarding and exit interviews
Skip-level-one-to-ones
Living up to our company values
Metrics around diversity and inclusion, if available; e.g. makeup of the organization
Performance and promotion rates across the organization to identify potential biases in the process, e.g. average time at a certain level, number of promotions per manager (this can point towards biases in the process), promotions across different groups and levels
A range of hiring metrics throughout the whole process, if I’m hiring
Perception of our organization with stakeholders
How are our employees doing? Where can we do better?
Developer satisfaction (SPACE)
Communication and collaboration (SPACE)
Access to information and knowledge
Employee engagement score
Qualitative employee feedback
Retention rates
How am I doing?
Results achieve - (that’s ultimately what I’m being measured on, typically in combination with
Employee, manager, stakeholder feedback
Efficiency measures
How predictable is our work?
Planned vs. delivered story points
Delivery delays, missed goals
How (well) are we learning? Are we repeating the same mistakes?
Overall visibility into the organization and our operations from a technical and team perspective (e.g. do we have metrics, which areas are we not able to get good visibility into, what’s our test coverage?). This is really foundational: We can’t improve if we don’t know how we’re doing.
Growth experiment results
Self-reported data and/or quantitative data on follow-through on action items from incident reviews/postmortems, team retrospectives
Patterns and themes in incidents
Self-reported employee data on feedback and learning culture
Time to onboard (e.g. time to n-th PR)
How well are we capable of making changes?
Lead time
Cycle time
Velocity
Deployment frequency (DORA)
Lead time to Changes (DORA)
Experiments run
Aggregate data of other metrics that I review: Trends overall
Are we making the most out of what we’re spending?
Focus across the organization (WIP, number of high-level priorities)
ROI of our initiatives
In most companies, I don’t have quantitative data on all of these aspects, or at least not at first - usually a big part of my job is to help an organization get better visibility into the various aspects that make it successful. And it’s not just about quantitative data alone, as a lot of the points above show: I find qualitative data incredibly important, which is also why it’s part of what I ask my direct reports to give me updates about every week.
My list of metrics is long, and that’s the point: There are a lot of areas that are important for me to keep an eye out for in order to develop a narrative about our organization.
It’s my job to constantly have situational awareness, i.e., a comprehensive view of my organization and can make confident statements about how we’re doing, our risks, and recommendations for priority investments - and decisions that benefit the company and help us achieve our goals.
I tried to break things down as much as I could here, because let’s be honest, measurement is so nuanced and based on everyone’s unique situation. So I want to leave you with a few final thoughts:
Context is key.
Selecting the right metrics for your engineering team depends on a variety of factors - company maturation, team size, where you sit within the organization - and involves careful balance and reflection.
Avoid overemphasizing one aspect, maintain a solution-oriented approach, and embrace transparency.
Metrics, when chosen thoughtfully, tested often, and shared openly, will become powerful tools for driving positive change.
With thanks to Maggie Litton and Riasat Rakin for the valuable perspectives they added to this article.
