Building Your Organizational Observability (OO11y) System: Exercises, Templates, and Your First 30 Days
This is the practical companion with self-guided exercises to my article about the OO11y (Organizational Observability) framework, based on a workshop I ran for Directors, VPs, and other senior engineering leaders at LDX3 London 2026.
Recap: The OO11y (organizational observability) framework
In the previous article, you got to know the OO11y framework, and here’s a brief recap:
Most leaders are monitoring their orgs (reacting when something breaks) rather than observing them (having systems that let you understand what’s going on and why). That’s a problem, and it’s getting worse.
Organizational observability rests on three pillars: Signals (what you measure), Conversations & Chats (what you hear), and Judgment (what you trust). You need all three. They’re joined through sense-making, the practice of cross-referencing your three pillars and paying attention to where they don’t line up. That’s where the real insight happens.
OO11y needs to be a system, not a function of your personal heroic effort. Distribute observability across your leadership team.
Start with one gap, not a complete overhaul. Resist the desire to know more of everything, now. Instead, begin with the most pressing areas and build out your OO11y from there. Put a 15-minute weekly review on your calendar to practice OO11y and tweak your system over time.
Article summary
This is the hands-on companion to the OO11y framework to help you fix the visibility gaps in your org, now.
The exercises in here will walk you through diagnosing your visibility gaps across Signals, Conversations & Chats, and Judgment, and synthesize them through the Sense-Making layer. Bonus exercises go deeper on leading vs. lagging signals, async offloading, signal integrity, and triangulation.
This is for you! While this guide is primarily written with the context of directors, heads of engineering, VPs, and people above those levels in mind, the principles of building systems to see clearly rather than relying on heroic effort or hoping your instincts catch everything are important at every leadership level. If you're an engineering manager, a team lead, or a senior IC moving into leadership, start building these habits now. Your future self will thank you.
My tip: Do this exercise with a peer! I heard from several leaders that they found it very valuable to talk through their findings and get ideas from each other.
The Extended Worksheet
Get the Extended worksheet
Download the OO11y worksheet (PDF, Google Drive link) for a printable version or make a copy of this Google Doc to use directly.
You can work through everything end-to-end (budget about 60–90 minutes) or do one section at a time over a week. If you attended the workshop, this expands on what you started there, with several bonus exercises I had to cut for time.
Exercise 1: Set Your Foundation (~10 min)
Before you instrument, decide what you're instrumenting for.
Pick 1–2 areas where you most need better visibility right now, and rate your current visibility on a scale of 1–6 (1 = flying completely blind, 6 = I can confidently say how this is going right now).
Write down why these areas matter most to you in your current context.
An important note as you get started: Resist the temptation to try and optimize for everything, immediately. Instead,focus on starting to get better, somewhere. The areas for which visibility matters most will change overtime and you’ll build a system today to adjust for that.
2. Analysis & Exploration
A. Signals Map: What do you already measure? (~10 min)
Signals are your quantitative indicators: dashboards, metrics, trend lines. A small number of signals you actually act on beats 50 dashboards no one reviews. Mind the gap!
→ For the priority areas you identified above: What signals do you currently get? Do you regularly ★ look at or → act on them? Which one(s) do you find most valuable? If you don't have any signals, what could you try?
Where are your signal gaps? For your priority areas, what do you not have a quantitative read on? (Examples: no data on how capacity is actually allocated vs. planned, no quality metrics beyond incident count, no engagement data between annual surveys, no visibility into cross-team dependencies.)
Remember: you need signals that tell you about productivity (doing the right thing, well), effectiveness (achieving goals, quality holding, building the right things), and efficiency (avoiding waste), not just raw throughput.
Bonus exercise: Leading vs. Lagging Check (~5 min)
Are your signals mostly telling you about things that already happened (lagging) or things that are starting to drift (leading)?
→ Go back to your signal list and mark each one:
L (Lagging): Tells you about something that already happened. Examples: missed deadline, attrition, production incident, post-mortem findings.
E (Early/Leading): Tells you about something starting to drift. Examples: rising WIP, increasing cycle time, declining PR review speed, growing on-call burden, dropping engagement scores.
If most of your signals are lagging, you're finding out about problems after the fact. What leading indicators could you add for your priority areas?
Also consider: do you have any emergence signals, things that would tell you the landscape is changing, not just that a known metric moved? A sudden increase in cross-team escalations, new types of requests from a team that didn't have them before, unexpected changes in who's reaching out to you.
B. Conversations & Chats Map: What do you hear? (~15 min)
Conversations & chats are your qualitative detail, context, and stories behind numbers. This "grapevine intel" includes scheduled conversations as well as chats (informal chatter like DMs, hallway encounters, + async exchanges like written updates). This pillar is under lots of pressure: less time for conversations, but more happening that only they can surface. Offload status updates to async to protect this time!
→ Who are your useful conversations & chats with? What do you learn? Who gives you info about emerging shifts?
Circle the areas where you're getting the least useful information. Pay attention to where you're lacking information about emerging shifts.
Most senior leaders cover downward reasonably well but are nearly absent laterally and upward, which is where the most surprising and important information often comes from. I hear so often that busy leaders don't want to bother their peers, but I think this layer is critically underutilized (more about working better with your peers here).
→ Who are your 2–3 irreplaceable conversations & chats with?
______________
______________
______________
Bonus: A bank of generative questions to get more out of your conversations
Conversations should be generative, not performative. Many 1:1s devolve into status updates, with your report reading out what their teams are doing, you nodding along, nothing new learned. A generative conversation surfaces something neither person knew going in. Here are some to try:
Down (with your reports):
"What's worrying you that you haven't told me yet?"
"What's the one thing your team is spending time on that you think they shouldn't be?"
"If you had to bet on where we'll hit a problem in the next month, where would it be?"
"How are decisions actually getting made on your team right now?"
More questions in this guide to running more useful skip-level meetings.
Across (with peers and cross-functional partners):
"What's changed in the last two weeks that I might not know about?"
"Where are you seeing dependencies forming that could affect both of us?"
"What are you hearing from leadership that I should know about?"
“What are your instincts telling you at the moment?”
Up (with your own leadership):
"What's shifting in priorities that hasn't been announced yet?"
"What pressures are you navigating that I should be aware of?"
"How are you thinking about [X]?" (where X is a decision or direction that affects your org)
Bonus: The Offload Audit (~10 min)
The goal: identify information you're currently getting from live conversations that could come to you asynchronously instead, so your limited live conversation time goes to what only a real conversation can surface.
→ List your recurring live meetings and conversations. For each, ask: could this information come to me async?
Examples of async alternatives: structured weekly written updates from reports (here's a template for that), team health pulse surveys, retro summaries that flow to you without attending every retro, shared dashboards where teams self-report status, Slack digests for cross-team syncs.
Pick 2–3 to shift to async in the next month. Use the freed-up time to add or protect an irreplaceable conversation.
C. Judgment Map: How confident are you in what you're hearing? (~15 min)
Corrupted telemetry is worse than missing telemetry you're aware of. A manager who reports green when things are red is more dangerous than a gap you know about. The question isn't just "can you see and hear enough?", it's "how much can you trust the information you're receiving, and, where it's limited, what can you do to increase it?" Do the people around you have a handle on things? Are they challenging assumptions? Factoring in risks? Making good decisions?
How confident are you in information you receive from people who are closest to your priority areas? For lower-confidence areas (<4), what could you try to increase it? (e.g., triangulating, interaction shift like coaching people in gap areas)
People may be unreliable narrators not out of malice (there may be exceptions, but that's all they are), but because of skill gaps, blind spots, fear, or a desire to look competent.
Bonus: Signal Integrity Deep-Dive (~10 min)
For each person or area where your confidence is below 4: What specific evidence gives you doubt? Not a vague feeling, but concrete observations. Examples:
A report who always says things are "fine," even when you can see signs of strain
A partner team whose timelines keep slipping despite reassurances
A boss who shares decisions but not the reasoning behind them
A manager whose updates are always positive but whose team's results tell a different story
Is the filtering a skill gap or a deeper problem? This matters because it changes your response. A skill gap is coachable: you can help a manager learn to surface risks proactively, or coach them on giving you more nuanced updates. A deeper problem (someone actively hiding issues, or fundamentally unable to assess their own area) requires a different intervention.
Bonus: Triangulation Inventory (~10 min)
For each low-confidence area, identify at least one additional information source that could give you a second perspective. The principle: don't rely on a single information source for anything that matters.
Triangulation tactics to consider:
Sit in on a team retro or planning session (not to direct, but to get a more direct read on how the team thinks and operates)
Have a direct conversation with a cross-functional counterpart about a shared area
Ask a skip-level: "How are decisions made on your team?"
Compare what two different people tell you about the same situation
Attend a design review
Look at the work product directly (PRs, docs, designs) rather than relying on someone's summary
Pick one triangulation action per low-confidence area and put it on your calendar.
Designing your OO11y system (~15 min)
Having three pillars instrumented is only the foundation: you need to build a practice to synthesize them together and catch discrepancies and dissonance. Also: you can't observe a complex, distributed system from a single node. Distribute observability across your leadership team.
What can you do to improve your OO11y?
Make it a practice, make it yours:
Find 15 minutes next week for your next OO11y review, put the time into your calendar. Review the weekly template below and add or edit what you need to make it work for you.
Your Weekly OO11y Review Template
This is the core of your ongoing system. Adapt it: add questions, remove what isn't useful, adjust the focus areas as your context changes. The point is to make it yours.
Monthly and Quarterly Check-Ins
Your weekly review is the foundation, but you also need the longer view to catch bigger patterns.
Monthly pattern conversation (with your leadership team, ~30 min):
What are we seeing across our areas? Any common themes?
What's changed in the last month?
What are we worried about?
Are our priorities still right, or has something shifted?
Quarterly reset (personal, ~45 min):
Have my baselines shifted? What does "healthy" look like now vs. three months ago?
Has my context changed? New priorities, new team members, new pressures?
What should I be observing differently?
Do I need to re-prioritize my foundation areas?
How is my OO11y system working? What's giving me good signal, and what isn't?
Getting Started: Your First 30 Days
You don't need to build the whole system before you start using it. Here's a realistic path:
Week 1: Set up. Complete the exercises above (or at least Sections 1 and 2). Put your weekly review on the calendar. Take your one smallest concrete step toward closing your biggest gap.
Week 2: First review. Do your first weekly OO11y review using the template. It will feel rough, and that's fine. Start closing your biggest gap. Have at least one conversation in a direction you identified as thin.
Week 3: Watch for dissonance. Second weekly review. This time, pay particular attention to whether your three pillars are telling you the same story. Identify one person you want to develop as an observability agent on your team.
Week 4: Start distributing. Third weekly review. Start a conversation with that person about building their own sensing capacity. Assess: is your biggest gap starting to close? Do you need to adjust your approach?
End of month 1: Hold your first monthly pattern conversation with your leadership team. Decide what's working in your OO11y system and what needs to change. If you haven't done the bonus exercises yet, now's a good time.
Bonus: Find an Accountability Partner
Building a new habit is easier when someone else is doing it too. Find someone at your level who's also interested in building their OO11y system (a peer director, a fellow VP, someone from the workshop).
Commit to checking in on each other's progress. Share: your biggest gap, your first step, what you're finding in your weekly reviews. I'd suggest checking in about 4 weeks after you both start.
The Key Principles To Remember
As a reminder to take with you:
Systematizing & instrumentation > reactivity & fire-fighting
Starting somewhere, now > doing everything, never
Protect your sense-making time, weekly
Evolve it. It's alive!
It doesn't stop because you're busy. It exists because you're busy.
Downloads and Further Reading
Download the OO11y worksheet (PDF, Google Drive link) for a printable version or make a copy of this Google Doc to use directly.
The OO11y Framework — the full conceptual framework this guide is based on
How to Push Information to Your Boss (with a free template!) — for setting up async status updates with your own boss, or to give your direct reports a structure for what to share with you
Managing Up: The Most Important Skill for Anyone Who Has a Manager — for building better upward and lateral conversations
Start this week.
