Join Shane Gibson as he chats with Timo Dechau about Metric Trees
Listen
Listen on all good podcast hosts or over at:
https://podcast.agiledata.io/e/the-pattern-of-metric-trees-with-timo-dechau-episode-77/
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
You can get in touch with Timo via LinkedIn or over at https://timodechau.com
Tired of vague data requests and endless requirement meetings? The Information Product Canvas helps you get clarity in 30 minutes or less?
Google NotebookLM Mindmap
Google NoteBookLM Briefing
Executive Summary
Metric Trees: A Framework for Strategic Alignment and Business Transparency
Metric trees are a powerful framework for visualizing the mathematical and logical relationships between key business metrics. Their primary function is to deconstruct high-level, often non-actionable, “output metrics” like Monthly Recurring Revenue (MRR) into a hierarchy of actionable “input metrics” that individual teams can directly influence. This transforms abstract company goals into a tangible map that guides strategic planning and execution.
The core value of a metric tree lies in its ability to serve as a shared language and a visual representation of the entire business model. It fosters alignment across disparate teams—such as product, marketing, sales, and data—by clarifying how their specific activities contribute to top-line objectives. By creating this transparent map, organizations can have more focused conversations about priorities, diagnose performance issues, and measure the true impact of strategic initiatives.
Implementing metric trees effectively requires a blend of product-thinking patterns and data discipline. Methodologies like Event Storming and Domain-Driven Design are crucial for identifying the core processes and high-value “heartbeat events” that underpin the business. Ultimately, a metric tree is a strategic tool for planning, communication, and high-level monitoring; it complements, rather than replaces, the need for deep-dive, explorative dashboards.
The Challenge: Isolated Metrics and Disconnected Teams
Organizations often struggle with a fragmented understanding of performance, driven by metrics that are viewed in isolation and teams that operate on different “planets.” This disconnect manifests in several key challenges:
• The Problem of Isolated Metrics: Stakeholders are frequently presented with long lists of potential metrics (the “PDF with 100 SaaS metrics” problem) without any context for how they interrelate. A metric like MRR, viewed alone, offers little guidance on what actions to take. As Timo Dechau states, the critical missing element is the “relationship” between metrics, which is essential for making them actionable.
• The “Iceberg” of Complexity: Metrics that appear simple on the surface, such as “Active User” or “MRR,” conceal immense complexity. Defining these metrics accurately requires a deep understanding of the product’s specific use cases and the business’s various edge cases, a process that can take weeks or months. A well-defined metric can “change the track for a company,” but a poorly defined one creates confusion.
• Siloed Data Cohorts: The data domain itself is often fragmented. Cohorts focused on the data warehouse, product analytics, and data science frequently use different techniques and technology stacks, even when working with similar data patterns. Product analytics, with its focus on behavioral event data and sequence analysis (funnels, cohorts), has historically operated separately from classic Business Intelligence (BI), creating what Dechau describes as “two different planets” that rarely interact.
Defining the Metric Tree
A metric tree, also known as a driver tree, is a visual framework that maps the relationships between metrics, showing how lower-level inputs drive higher-level outputs.
• Core Concept: It functions as a deconstruction of a primary business goal into its constituent parts. Every metric has a relationship to another, and the tree makes these connections explicit.
• Primary Function: Its purpose is to translate a high-level, non-actionable output metric into a series of actionable input metrics.
◦ Output Metric: A lagging indicator that reflects past success (e.g., Revenue, Profit). It is difficult for a team to influence directly.
◦ Input Metric: A leading indicator that teams can directly control (e.g., New Accounts, Conversion Rate from Trial).
• Structure: The tree is a hierarchical model that can often be expressed as a mathematical equation. For example:
◦ MRR is composed of New MRR + Expansion MRR - Contraction MRR - Churned MRR.
◦ New MRR can be broken down further into New Subscribers * Average Plan Price.
◦ New Subscribers can be derived from New Accounts * Account-to-Subscriber Conversion Rate.
This decomposition continues until the metrics at the lowest level of the branches are things a team can directly execute against, such as running more webinars to increase New Accounts.
The Strategic Value of Metric Trees
The primary value of a metric tree is not just in the metrics themselves, but in the clarity, alignment, and strategic conversations it enables.
A Shared Map for the Business
The metric tree acts as a universally understood “map” of the business operating model.
• Creates a Common Language: It allows different departments to point to the same part of the map and understand how their work affects others, breaking down communication silos.
• Fosters Transparency: It makes the mechanics of the business model clear to everyone. For many employees, it may be the first time they see a clear illustration of how the company generates revenue.
• Reveals Interdependencies: The map highlights how initiatives in one area can impact metrics elsewhere. Dechau notes that workshops to build these trees often lead to “eye-opening” moments where teams realize their actions might be inadvertently hurting another revenue stream.
Driving Action and Measuring Impact
Metric trees connect daily work to strategic goals, making it easier to plan and measure initiatives.
• Connects Strategy to Execution: Teams can clearly see their area of influence on the map. A marketing team knows its efforts to generate new accounts are a direct input to the company’s overall revenue goal.
• Measures Initiative Success: A specific metric tree, or “sub-tree,” can be built for a new initiative. This allows the team to define success upfront and provides a “control instance” to validate whether local efforts (e.g., A/B tests) are creating a meaningful impact on the larger business goals.
• Identifies Opportunities: By populating the tree with data, teams can spot areas of high potential. For instance, a part of the funnel with high volume but low conversion rates becomes an obvious target for optimization.
“If a metric is not actionable, it will have a hard life. It lives lonely on this dashboard and no one has an idea what to do with it.” - Timo Dechau
Implementation and Good Practices
Building and using a metric tree is a strategic exercise that requires a structured approach and an awareness of its limitations.
Starting the Journey
1. Map the Process: The first step is to gain a deep understanding of the customer journey. This is best achieved through a collaborative workshop, like an Event Storming session, involving people from all relevant disciplines who can map the process from initial awareness to long-term retention. This process naturally identifies the key milestones and potential metrics.
2. Align with Strategy: It is critical to sit down with the leadership team to understand their strategic priorities for the next 6-12 months. This alignment ensures that the initial metric tree focuses on what is most important to the business, which dramatically increases buy-in and adoption.
The Art of Event Tracking
A robust metric tree is built on well-defined data. This requires moving beyond simplistic interaction tracking to a more meaningful model of product usage.
Tracking Interactions
Capturing every click, scroll, and granular user action. This is the “take everything and look at it later” approach.
A high volume of noisy, low-signal data that is difficult for human analysis and disconnected from business success.
Tracking Product Usage
Applying Domain-Driven Design to identify core business entities (e.g., Account, Subscription, Project) and their lifecycles (e.g., Created, Updated, Deleted).
A small set (~15-20) of high-value, meaningful events that directly reflect product usage and can be used to build core metrics.
A key goal is to identify the “Heartbeat Event”—the single, central event that proves the product is alive and delivering value. For Slack, this is sending a message; for Miro, it’s adding an asset to a board. This core event can often be used to define multiple key metrics.
Practical Considerations and Limitations
• Keep it Simple: Avoid the temptation to “boil the ocean” by mapping every conceivable metric. An overly complex tree with 90+ nodes becomes an un-operational “monster.” It is better to start with a simple model and use sub-trees for specific initiatives.
• Acknowledge Timelessness: A standard metric tree exists in a timeless space. It does not inherently account for the time lag between an action and its result (e.g., an increase in new accounts may not impact revenue for 60-90 days). This requires separate cohort analysis, which does not fit the tree structure.
• It’s Not a Dashboard: A metric tree is a tool for planning, communication, and high-level monitoring. It is not designed for deep-dive, explorative analysis to find the root cause of a problem. Dashboards are still required for that function.
• Who Does the Work? The skillset required—combining product thinking, data expertise, and strategic facilitation—is often found in senior data leaders. Heads of Data are well-positioned to lead this work, as it provides them a strategic lever to connect their team’s efforts directly to business value.
The Role of North Star Metrics
The concept of a North Star Metric is closely related to, but distinct from, the top-line metrics on a business-focused metric tree.
• Correct Definition: A North Star Metric is fundamentally tied to successful customer experience. It measures the value customers receive from the product. Revenue is the result of delivering this value, not the value itself.
• Common Pitfall: A frequent mistake is labeling a revenue goal like “New MRR” as a North Star Metric. This confuses an internal business outcome with customer success. Shane Gibson jokes these should be called “South Star” or “East Star” metrics instead.
• Relationship to Metric Trees: The North Star Metric is an ideal candidate for the top of a product-focused metric tree, with input metrics below it defining the user behaviors that lead to a successful customer experience.
On North Star Metrics: “The concept of a North Star metric is it’s always bound to successful customer experience, it has nothing to do with revenue. Obviously we hope when people have a good customer experience that... it has a causal connection to more revenue.” - Timo Dechau
Future Outlook: Metric Trees in an AI-Driven World
As technology evolves, the principles behind metric trees become even more critical for managing complexity and providing essential business context.
• Context for LLMs: A defined metric tree provides a structured map of the business model. This is invaluable context for AI agents and LLMs, enabling them to perform more accurate “what-if” simulations, brainstorm strategies, and answer complex business questions.
• Governing Democratized Development: The rise of AI will enable the rapid creation of thousands of small, single-purpose applications (”AI slop apps”). This will create immense data complexity. A framework of core concepts (e.g., a universal definition of “user”), governed events, and a central metric tree will be essential to ensure these new applications contribute to meaningful business goals rather than creating data chaos. The metric tree provides the “principles, policies, and patterns” needed for this new landscape.
Tired of vague data requests and endless requirement meetings? The Information Product Canvas helps you get clarity in 30 minutes or less?
Transcript
Shane: Welcome to the Agile Data Podcast. I’m Shane Gibson.
Timo: , And I’m Timo Dechau.
Shane: Hey, Timo. Thank you for coming on the show today. I would like to talk about metric trees, something that’s intrigued me for a little while, so it’s great to have somebody who knows a lot about them on the show. But before we rip into that, why don’t you give the audience a bit of background about yourself.
Timo: I didn’t start out in data. I don’t know. I, it is interesting question would be if anyone actually ever started out in data. I started out in product. I will give some connections to metric trees later. Why? It basically was really important for me to connect it with product. . I started out on product was very quickly annoyed by people doing features just by gut feeling. I wanted to have a different kind of layer, so therefore I started to introduce analytics data to it. So this was really the early days. It was not even called product analytics at that point of time. The term came later. , So I spent a lot of time while walking through different kind of product roles to always setting up data setups for that. Also did the first connections there with data warehouses. Just a slightly little bit. And then after eight years in product, , I decided to go all in data.
So it was more focusing on the analytics side of things. So I’d say classic marketing analytics and product analytics. At that point of time, it was the high time in Berlin where a lot of e-commerce startups were really growing a lot. And so I was doing first data warehouse setups mostly in the marketing analytics space. Like people wanted to have their own attribution models and these things. I did this I spent then a lot of time in this kind of area. And then at some point I thought, okay, I want to go back to the roots and want to do more product analytics again. But I wanted to do product analytics in the data warehouse, , which required some things that were not so common at the time.
So one is like an event data model, which some people had some experience with, but not. Potentially for product use cases. , And then the second thing is there was no approach, like Amplitude postdoc were the classic tools that you do product analytics, they then were possible to put them on top of a warehouse. But then I figured out actually there, most of the time they’re too complicated for just doing product work. So in the end, you have to have a metric first product analytics approach. And this is where I came back to metrics where I never really had a good relationship with them.
So we might talk about this in this session. But I had to basically revisit my relationship to metrics. And this is where I came across metric trees again. I had them already at university. And yeah, starting to dive deeper into this.
Shane: , I’ve followed your stuff for a long time, so thank you for sharing so much great content. And what’s always intrigued me is in the data domain, I could almost see a data warehouse cohort, a product analytics cohort, and a data science cohort. And , although each of those cohorts are using very similar patterns, very similar techniques.
They always seem to use different technology stacks. And then we had the whole, what was it? Composable, combustible, whatever it is. Yeah. Yeah. So that idea that you don’t need a dedicated product analytics tool. You can just do it off your cloud data warehouse or cloud data warehouse database.
So it’s really interesting that there was almost a separate track, wasn’t there for product analytics and the techniques you used and the technologies compared to some of the other, cohorts or tracks in that data domain.
Timo: It is super weird because like sometimes you talk to people in the data space. So let’s say who do classic bi and , this was my problem when I was starting out. So because I was looking for specific way how I can model product data, the same as marketing data. Marketing data is a bit less, but like it’s a lot behavioral data.
So a lot of event data. And you do sequence analysis so funnels or cohorts and something like that. And it was really a hard time to talk to people who are in the classic BI space and tell them . Most of the data models don’t really work for me because I have to do sequence analyze.
So I cannot create 200 fact tables. It would be a bit crazy. And , I really had a hard time to explain, but it was also interesting because I learned a lot from them. I think some learned a little bit from me but it’s interesting because everything calls data, but I always call it they live on two different planets and they rarely visit each other.
I think it’s getting more, it’s getting better. So I think there’s more conversion now and something which I, for example, work a lot on. So my last, let’s say my last mission, maybe it’s, if it’s my last, I don’t know, but my biggest one is really crossing the whole marketing and product behavioral data with revenue data which then makes it really interesting if you do this.
Shane: We had a customer and, they were a scale up.
And so we needed to do some work around . Standard metrics for a startup. The pirate metrics, a r, all those kind of good things. And when you first go into those areas, you go, how hard can it be? And then you find out
And it’s damn this should be a solve problem right now.
The core metrics for startups, should be well-defined. They all use, standard type of software as a service products for capturing subscriptions and signups and, all those kind of things. There’s three or four of them that everybody tends to use. It shouldn’t be hard, it should be patterns out there that you could just implement and make it really easy. And again, whenever you say those words, you know it’s gonna bite you in the bum and then you start doing it and you’re like, actually for some reason there’s a whole lot of complexity just hidden under the top of that layer that bites you.
Timo: I just do MRR calculation. So it’s let’s say technically these are all the same metrics. So there’s, let’s say a whole metric set around MRR. But the tricky thing is , how do you define it under the hood? Because everyone comes to you and in every case, company tells me like, no, we have a very standard subscription model.
There’s nothing crazy in it. And then I always know, okay, yeah, this will hold for two weeks until we do the first investigation into all the edge cases, and then we will just cover all the edge cases. Ah yeah, we, there we had to do something different. Oh yeah. And then yeah, we had this switch from one subscription to the, and so it’s always messy.
And that’s the same with product stuff, I think the pirate metrics are still a really good framework to understand how product, and let’s say this business around is evolving. But let’s say just go for something like retention or active user or activated. So activated by default is in the end one metric, but it can take you weeks and sometimes months to redefine how you basically calculate this because it really comes down how the product works, what kind of use cases are you actually solving.
So it’s a fascinating, interesting thing, but it’s also good ‘cause in the end it’s one metric that you just see, but it’s an iceberg. So on the top you just see this one metric, but quite complex underneath. But this is where the fun is. If you really do a really good job there and the definition and so on, it can really change the track for our company.
Shane: Even then though, there’s still outside of the metrics, there’s some core patterns you’ve gotta decide from an engineering point of view. So again, when I look at product analytics, when I look at event tracking in a software product, typically you have a choice of pull every event. so effectively pull event logs to say lots of things happened, and then apply the events you care about after the fact.
So more of a Lakey type pattern. Or only define the events you want in the product when at the beginning. And then only bring those in and use those. And if you want to look at a new event, you’ve gotta go and define it again. And there’s a trade off there, right between this,
If you get every event, you get this wash of noise and it’s really hard to figure out where the signal is.
But when I worked with a lot of product teams, they seemed really reticent to define the events that were important. It was easy for them to say, I’ll just take everything and we’ll look at it later, versus this is the event that will tell us that this feature was successful.
That’s actually quite a hard metric to define,
Timo: it is, this is this is another this is my second passion topic and this is the one where I wrote a book about. I think the problem is when you go into this, you have to distinguish between tracking interactions and tracking actually product usage. And so most of the setups are tracking interactions.
Where people click, and it’s quite understandable why you do this because you have to define something. So it’s not it’s not everyone’s job to define what you want to track. So it’s usually a side job that you have to do in between. So the, let’s say the most approachable way to do this is you open up your own application, then you see, okay, where can people actually do something?
And so they’re like, okay, they can click here, they can click here and they can do this. And this is what I mean you track interactions. If, let’s say, if this whole output is just for machines, then it might be. Okay. We are still not there. That, let’s say you can collect all these, let’s say very noisy, very granular data, and then you have something running over it, finding some patterns.
So far I didn’t really see things that go in this direction, but it might come at some point, but when a human has to analyze it, this doesn’t work. It’s far too far away from what actually defines product success or what actually defines business success. So it has too many noise, as you said, and so therefore what I ask product teams is define the use cases and define the jobs to be done.
Define the entities that make your product. So in the end, it’s classic domain driven design, what I do with them. And so okay, let’s define the different kind of entities that build up your products. It’s usually five or six. And then we define how does the lifecycle look into this? So let’s say we have an account can be created, can be updated, can be deleted. So three events updated usually you don’t need because there’s not often a business value in it if someone update an account. So who would optimize on account update updating usually. Yeah. Any there edge cases. But, and you can do this for everything else. You can do it for subscriptions. You can do it for whatever your product is built off. And then you can just end up with. This is usually my take. You end up with 15 events and the same, you end up with 10 metrics. It’s sometimes really quite crazy if you really have a good setup. So right now, I was just working on a project where we had one event that was defining six core metrics that could explain if the business is running or not. Because there’s always, this is always my interesting take when I work with the client is in every setup you have something which I call a heartbeat event. And this is this is a central event. If you just need to track one event, this is the one that tells you is your product still alive or not.
So for Slack, messages for Miro, if let’s say an asset, it’s added to a board or so, you have always this one event where you know, okay, when this is still coming in, things are good, we don’t know about it, but they’re still there looking good. And you have to get to this level, then you can basically tame the chaos that product analytics can cause.
Shane: It sounds a bit like a North Star metric. This idea that yes, all the other things are important, but actually if you can define the one thing that really is the core, focus on that and the other things are useful, but if you don’t focus on that one thing,
Timo: I always have to fight a little bit for the North Star metric because there are so many definitions out there, and I think the worst case is when someone comes to you and says yeah, new MRIs or North Star metric where you say, no, it’s actually not the, let’s say the concept of a North Star metric is it’s always bound to successful.
Customer experience, it has nothing to do with revenue. Obviously we hope when people have a good customer experience that let’s say it, it has a cau connection to more revenue. And then the North Star metrics is usually also built up by two or three input metrics that lead to it. But in the end, it’s the same what you said.
You have a set of three metrics that could explain how is your product actually delivering value at the moment.
Shane: I’m with you is that, the North Star metric is success for the customer, not success for the business because if the customers get the success that you promised them, then your business will grow. But yeah, time and time again, I see North Star metrics being internal metrics. And it’s maybe we should just go read the definition again or call it something else, south Star Metric or East Star. Like just give it a different name. That’s okay.
Timo: Yeah. You can always call us like this is our, yeah, I don’t know. I think North Star Metrics is just a great name, so it’s a great label. So you lo everyone understands what it is. It sounds great, like adventure. So this is our North Star. We have to go there.
Shane: That’s my standing joke is that as data people, we laugh about the fact that our stakeholders can’t define active customer, active , subscription. Yet as in the data domain, we argue about the definition of a semantic layer, a data product, a north star metric, day in, day out.
So we’re as bad, if not worse than them. So on that note let’s go and talk about metric trees. So if somebody said to you, what the hell is a metric tree? How would you describe it?
Timo: First of all the same. I think it might be also a definition problem. So some people call it driver tree. There’s also like the term of kpit. I’m not really good in definitions. I know that they have slightly different variations of it. So I give you my explanation and my definition of it. So one problem, what you often have with metrics is when you take a metric alone, so let’s say you have a software as a service. So you’re not really sure what you should measure. You are on LinkedIn. One of this posts where someone is posting, Hey, I just compiled the greatest collection of software as a service.
Metrics, like common metrics, and you get my PDF with this 100 metrics. These things never really work for someone. And the problem is why they don’t work. It’s because everyone picks then a metric and looks at it in isolation. So I don’t know. Yeah. For example, MRR, monthly recurring revenue. So what is you have a standard alone there and then you don’t really have an idea what you should do with it.
Because two things are missing and what defines the metric? We mostly is it’s missing a relationship. So let’s say every metric has a relationship to another metric. They have I cannot even think about one, which is really standing alone. So usually you have a relationship and why do you need this relationship? Because not every metric is very actionable. So we talked about this North star, when you say, Hey, revenues are North Star. So the problem with MRR monthly recurring revenue is. It’s an output metric, so it’s something that comes out at the end. So when the CEO goes into a meeting and let’s say assembles all the heads of the different teams and says we have to increase MRR, everyone would say, yeah, sure.
Sure it’s our business model, so it makes sense, but no one would immediately come up with an idea, oh yeah, sure, we should do that to increase more MRR. No, you usually break it down. So you break it down into, oh, okay. When we need MRR, and maybe we need more accounts because they could end up in a subscription and then this would be new MR. But then this is interesting because this is already the first step to build out a metric tree because then you have MRR, and then you say, okay, what is actually making up MRR? New subscribers multiply it with an average. Plan price or average price, and then new subscribers you can build up from new accounts with a conversion rate. And then you get into your first version of a metric tree. And I think the nice thing about a metric tree is that it explains how you can get from something that you will end up with to something that you can directly influence. So if you go to a marketing team, tell them , Hey, we need more new accounts, they usually have an idea what to do. So they will like, oh yeah, okay, maybe we have to run more webinars or we have to invest in a podcast or whatever. And so there you make it actionable. And at least this is my driving force. Why metric trees are interesting because they help you to break down something that you want to achieve to something that you can actually do. And this is the tricky thing. So if a metric is not actionable, will have a hard life. It lives lonely on this dashboard and no one has an idea what to do with it.
Shane: So if I think about it what you’re really saying is there’s a bunch of metrics and they have a relationship to each other.
And we treat it like a tree. So if we think about an HR human resources org chart where we typically see a tree of people and people report up this tree, what we are saying is if we can find the relationship metrics that behave that way, so these metrics support this other metric which supports this other metric, then that relationship helps us work out what we need to change in our business to move one or many of those metrics.
Is that kind of the concept?
Timo: Yeah. That’s the concept. You can also use it for that. Where I like to use it is to really try to explain how the business works on a, let’s say, common concept that at least the data team understands and that the business team understands. And this is something which I would say the metric tree so far worked best.
So I tried different kind of formats to bring both teams together. We did it. Okay, what kind of events do we have to measure? Usually it doesn’t really work well because it’s too abstract and, You sometimes find common ground, but it’s too far away from what people really care about. If you really build up a metric tree, you have really interesting conversations with, let’s say, when I do these workshops where it’s usually with startups where it’s easier to get relevant people into one room.
So then we have people from all the different kind of disciplines. And we have really good conversations because for a lot of people it’s for the first time that they see how actually the revenue is coming together in the company. So this is let’s say often, unfortunately for data people, it’s the first time that they see, oh, this is how we calculate revenue. Because it’s usually one person who does this part, but not everyone and then for products the same, they’re like, oh, okay. I was not aware that we also do this to get revenue. So it’s a really interesting exercise that can, open up a lot of black boxes and understands how the whole system’s actually building up. And then. Obviously, if you go a little bit more down, then also, all the different teams might find a place in this tree where they say, oh, this is actually where I work hard for the data team. They usually don’t find a place there. But they’re the ones who can build the metric tree and can provide the metrics. But let’s say you can have a part where you can see what product can do, you will have a part where you can see what sales do. Then you will have a part where you can see, oh, this is actually the influence area of marketing. And then in theory you can analyze how these different kind of levers that you have, let’s say on a lower level, how they’re actually attriting to the whole final output metrics. Every time I did this kind of format in a workshop, it was always some kind of eye-opening in there where someone said, oh, I never looked into this. Or someone even said actually right now, I think with some initiatives we are starting to hurt our revenue and so on.
No, no one has recognized that because now we see it for the first time that when we do this, let’s say this stream of revenue will have a problem, which no one really thought about. And so this is something where metrics we definitely can help to bring transparency into. Let’s say the mechanics, how the business actually making money that often is completely overlooked because a dashboard in theory does it, but it doesn’t have the structure.
The tree structure is nice because , someone can look at it and immediately understands what it does. Yeah. So go from the bottom up to the top or the other way down.
Shane: I think that’s part of the core value of it, is that simple map. ‘cause what we know is we know we, there’s a lot complexity and when we draw a simple map, a simple diagram, two things happen. People can visually ingest it and understand it. There’s something about a human looking at a map that just naturally sees the pattern.
And the second part is you can point to part of the map and everybody knows they’re talking about the same thing. So that idea of different parts of the organization, different silos, understanding how they affect another metric. So you know, your example, how do we increase monthly recurring revenue while there’s a bunch of metrics that the marketing team can affect, number of ads placed, number of prospects found optimizing the funnel from prospect to signup.
And they can look at that part of their map and go I can do some work in there. And that in theory will increase our monthly recurring revenue, because that’s up the tree. So that idea of a map gives us understanding and also gives us a clarity of a shed language, i’m focusing on this part of the tree.
Timo: Yeah, I’m very happy that you come up, that you say the word map because this is actually what it is. It’s one map that can be very useful. So at least when I create one for the projects, it’s not that we constantly work with it. So we might get to this point like how operational are metric trees, but where I always use it is when we come together and discuss big things.
Okay, what kind of metrics we should focus on? Let’s say what kind of initiatives we should look into. So this can always help to have a then sitting around and when we say, okay, look, now we want to focus on this kind of area, you can always look then in the, let’s say in the tree.
And the nice thing is when you have the tree with some data, then you can see, okay. We focus on this kind of area, , because we have the feeling we have a lot of volume here. But we can see for example, the conversion rates in this kind of areas are not really high. So there’s, let’s say there’s quite nice potential for us to improve things just slightly a little bit, but see a big impact because we see that a lot of push comes in there. , And so then we can analyze how this works. For that, it’s really nice to have it because again, everyone immediately knows where we are. So it’s not that you have too long. Explain how this might have an impact on revenue, because immediately everyone sees that
Shane: The good thing for data teams is it means they don’t have to go look at yet another stupid strategy PowerPoint with four boxes that have no context and no understanding and no data.
You might have to, but at least they can say that, that part of the pyramid or what are we doing this week? The circle, whatever the latest consulting picture is for for selling a story.
How does that map to the tree? One of the things you said though was data teams would struggle to figure out which metrics are theirs and that, that kind of gave me an idea. So, you know, One of the things I work on a lot is this idea of an information product, and one of the key things I teach teams is focus on the action and the outcome.
If you don’t understand what action the stakeholder’s gonna take and what outcome’s gonna be delivered from that action and the value of that outcome, then really there’s a risk. You’re doing data work that has no value. While we say stakeholders, were accountable for that actually as data teams, we should be as well, we should be holding our stakeholders to account that actually they can describe the value, at least the outcome,
That’s been taken.
So what would be interesting is if there’s a metric tree in place, a data team, which should be able to point to the metrics that they’re doing data work for or with to improve, i’m working with the marketing team to use data to reduce the time between a prospect being identified and signing up for an account.
And so they should actually be able to use the metric train to show where the data work is adding value to the organization in conjunction with those stakeholders, those different business operating groups,
Timo: yeah, exactly. I think this is where it can really play a really nice role because can sit in these, let’s say you work with the marketing team. Marketing team does some planning for the next two or three months. So they have some ideas let’s say they come up with three initiatives, what they want to do. And so you can take every initiative and you can then say, let’s say, okay, here on the big company metric tree these initiatives. They’re trying to improve these kind of areas. And then we can use this metric there as always, as our our control instance to really see, okay, do we see some impact? Because that’s a tricky thing. You can run a lot of initiatives and locally looks really great, but when you look at it on the big picture, everyone is yeah, I don’t know. AB tests look great, but I don’t see any kind of uptick somewhere. So it’s definitely nice to have, okay, this is the one big metric that it should, let’s say influence and let’s see if we can fire it up enough that we can see some influence. Then what you can do. The second thing is let’s say you can create different views of metric trees. So you can do this very high level for a company, but then you can also take every initiative and just build a metric tree for every initiative.
This is something which I often like to do because again, it helps you to understand, okay, how do we measure the success of this initiative? So you might have this one big metric that we identify in the company metric tree that let’s say is at the top end. And then we break it down for this kind of initiative.
And so then it really comes, let’s say you want to improve conversion rate. So then we can build the whole surrounding around this conversion rate and maybe even break it down by three different channels because this is what we are trying to achieve. You’re trying to get more people from this kind of channel.
So then you basically bake strategy into the metric tree and the strategy connects. Directly to the initiative that marketing is driving. And then everyone has a very clear idea what you do. And then also you will build something that in the end, can marketing and you can use to explain if this initiative was successful or not. , And you can validate it before you can ask people, okay, look, when we deliver this, does this make sense to you? When, let’s say when the initiative is over in four weeks and we report on this metrics and we define, okay, success looks like when we move this part, I don’t know, by 10%. So does everyone agree with that? And then because I have the feeling it makes it for a lot people easier to think in that way. So that they not don’t say, yeah, I don’t have really idea if I should agree or not to this because at least, yeah, I think metric, at least they understand. They might ask, okay, how do we define this kind of metric? But that’s fine. And so then we can spend some time to say, okay, how we define it. But I like this approach a lot. Also like to not always see the metric tree as this one big, okay. We explain the whole business model, but really to use it to explain how I run a specific initiative. It’s also nice for me. Let’s say personally when I work on these, let’s say, as a supporting part for data for these initiatives, it’s a great brainstorming tool for me. So let’s say someone comes up with this initiative, I do a first version of a metric tree. Then I say, does it really represent what they’re actually doing there?
And then I say, yeah, to a very generic part, but maybe not customized enough. So I’m always trying to tweak the metric tree that the people who run the initiative immediately find it in there. So let’s say I do something for e-commerce. And the e-commerce is really pushing and getting high loyal customers. Let’s say , they really want to improve the segment of high loyal customers who buy all the time. There’s these, let’s say, high buyers that you sometimes have. So when I just report on let’s say new customers or returning customers. Then I don’t really have it covered because yes, we are going for returning customers, but for a special segment within returning customers.
So therefore what I can do is I can say, okay, I break it down in three groups and new returning and let’s say VIP customers and then with the VIP customers for the first time, I make it visible what kind of impact the initiative right now has. Or let’s say I can see, okay, how is the share of these users is growing or not growing or whatever. And so this gives me a lot of, playroom to build something that can really support what the business is trying to do, but do it in a language or in a way that at least most of the people understand.
Shane: All righty. So I’ve got a shit ton of notes right now. So let’s let’s go through them one by one, because there’s so much gold in there. So the first thing you pulled out is this idea that actually metric trees is a shared language. Yeah. So you can, you’ll define a metric tree for what you think you heard from the organization of how their business operates.
And by putting that map, by putting that tree in front of them, they’re gonna identify where you’ve got it wrong. They’re gonna look at it and go yeah. Oh no, hold on. We don’t do that. Or we are different or, I don’t understand. It becomes that visual shared language of an entire business and their operating model
Timo: it maps the process to some degree, and that makes it easy for people to see, okay, is it actually what we do?
Shane: Then the second thing is you said when you want to go and do something new, you wanna do an initiative or some investment or change a process or go into a new market, you can pick the metrics you think you are gonna impact and you can guess how much you’re gonna impact them by. I remember many years ago, it was probably late nineties, early as two thousands, I got into the whole balance scorecard thing.
I was working for a vendor balance, scorecards were hot. A couple of ‘em were trying to build software. And one of the things was, you had this metric and effectively you could put a budget on it. We are gonna increase that by 10% over the next quarter. Now the software itself was pretty hokey.
But it was the conversations around what are we doing and how’s it gonna influence that metric. So that’s one of the key things you called out , is by saying that if we do these things, let’s guess which metrics are going to change, is that what you’re saying?
Timo: Yes, exactly. And then also really make sure that you measure this metric in the right kind of way. So that, for example, let’s say you have a corridor that you can see, okay, is this a normal movement of the metric or not? Or that the initiative is big enough that it can move the metric. This is a tricky thing. But often. At least when I work in product or marketing is that often you see a lot of initiatives that are just, let’s say they’re very tiny bits. ‘Cause I don’t know, they’re not really bold. And then it, you will not see anything. And you can even tell this before, okay we try to optimize 5% of the typical audiences that we get in there. So we are working on this. There might be still a strategic reason for that, but then you can still make it clear. It’s okay, so we work on that. But because , let’s say the sample or let’s say the audience is so small, we will not see an impact when we look at the completely blended global metric. So then, for example, you have to break down this metric maybe in different kind of path that you can still see something. But this check-in really helps to see, okay, how do we actually want to see if it makes an impact or not? ‘cause sometimes just because of the setup, you won’t see anything. You can already know this, that it might not happen.
Shane: And that was one of the problems with the balance scorecard. So we had this idea of cause and effect, we said that, I think you call them input and output metrics, but this idea that if I improve this metric on the bottom or the left then it has a cause and effect with a metric above it or on the right and.
Back then, we really wanted to find correlation, we really wanted to say well, actually, if we just throw all the data at the machine, the machine should be able to find causality or correlation between the metrics, I should about to codify this. It should be accurate. And we never got there.
We still don’t have the technology or the patterns to do that at the moment.
Timo: no. I don’t think so. So there, this is definitely not my area of expertise, so I know some people do some things like that, so they try to find out, , because you have some relationship when you build a metric tree that are not deterministic. So let’s say a classic metric tree, you can write as an equation because it’s basically that’s also quite nice that you can do it. But sometimes, for example, let’s say you have a metric which is called active users. And so you have a very well, well-defined active user definition where you say, okay, the people don’t just show up, they also do valuable stuff within your product. And so once they do it, and they do it within the last 30 days, we basically flag them as active users. So there’s definitely a correlation between active users and at some point starting a subscription, let’s say you have a free plan, and so you have an active user and a free plan. So there’s a correlation between the both, because you might assume, okay, someone has to be active to end up in a subscription. But let’s say it’s, it’s it’s not a direct connection, so you cannot really say, okay, whenever we get someone as an active user, we have a probability of, I don’t know, 57% that they will end up in a subscription. I guess this is more straightforward to calculate, but even there, I never really, came across, let’s say, super good models that can , predict this very good.
If someone knows this, please let me know. Ping me on LinkedIn. I’m always interested
Shane: Now the answer’s just AI slop, right? You can just AI it
Timo: Yeah
Shane: and maybe you can, maybe it’s good, that non-deterministic patent matching will be really good at this. I dunno, I haven’t tried lately,
Timo: So if we go to this, what you had before, like we, we create a lot of noise and we might throw it in and we might find something yes. That’s, yeah. But I think for correlation, analyze is potentially not,
Shane: but the key is the. Visualization of a map, the conversations of what on the map, the conversations of what you’re doing to improve the numbers on the map, the relationships that actually is where the value is. So yes, we could get programmatic deterministic correlation across the metrics. That would be awesome.
But actually the conversations so the next one is, our natural reaction is to boil the ocean.
If I think about an information product that’s lifetime value. I tend to say to organizations why don’t we break it down? We know that to do a lifetime value model, we actually have to have revenue cost to serve churn, there’s a whole sub submodel sub products that have value. Why don’t we build the revenue model first? . And give you a revenue information product. Why don’t we do the cost to serve second, and then over time we’ll get to that lifetime value. I can imagine with metric trees, the natural reaction for some data people is to define every metric.
So draw the map end to end. Define every metric before we even start doing anything. Where for me I like the ability to change fast. So for me, I would be going maybe sketch out a map really quickly as kinda like a blueprint of what we think , it looks like, and then focus on some metrics and define them and build them and deliver them and monitor them and use that information.
And then. Kind of color in the map over time as you learn more, ? How do you do it?
Timo: so I had a phase where, let’s say I was in the rabbit hole and I did one exercise where I was trying to map my whole content production as the most extensive metric tree possible, and it was, I still have it somewhere in Miro. It was really a monster. It was like no one could ever implement this. I think it had in the end, I don’t know, 90 nodes. And obviously this doesn’t work, so I could easily see that. It was a nice exercise, was a nice thing to do over the weekend. But it’s nonoperational so it doesn’t provide any operational value to do this. And by the way, it would not even possible to track the whole thing because it would include a lot of attribution stuff, which is not possible. So the same, what you say, I think like the model that I used now is. Try to really keep it simple. I don’t know. Let’s say also try more to work with sub trees and don’t be too, let’s say don’t be too deterministic with it. So give yourself the liberty to say, Hey, I create a, let’s say a specific tree for this kind of new product feature that gets introduced and have no idea how it connected to the other tree.
Now that’s fine. I live with that because local optimization is still better than no optimization, so therefore be nice to yourself. But no, you definitely have to keep it. Quite simple and then you have to know what you can do with it. So I know that some people use it for root cause analysis, and I think for that it’s really quite nice because I could you, you can say, okay, look, we lost so much new revenue let’s say, compared to last month. So can we follow up with the tree to really see where do we see where stuff got off? So it can help you to, let’s say, get a starting point, but then still, the deep dive is something different. The metric tree will not tell you where it happened and then. Another thing that I discovered was, is often overlooked, especially when you talk about metrics and the metric tree is, it’s really big risk is you have time is completely out of this thing. So the metric tree lives in a timeless space. So the problem is for example, you increase new accounts in the example that I had before. And then you wonder, okay, why don’t I see any kind of new revenue? Yeah. Because it happens in 60, 90 days. So whatever your model is or whatever the average time is that people usually take to upgrade into subscription.
So the metric tree always does a very bad job where things, let’s say down in the branches are already impacting and you see the impact three months later at the top and you have no idea where it’s coming from so if you would do it properly, you would do a cohort analysis where you cohort everything that happens something.
But this doesn’t work on a tree. I think the important thing about working with trees is really to know the limitations, whether it’s great or not. As I said, for me, mostly for planning, for brainstorming, for communication. I guess some people use it also. For these check-in meetings, let’s say have a weekly meeting, you check quickly, okay, how’s the business performing in specific areas? I think that can work as well. But it’s not like that. Some people thought, oh, get rid of the dashboards and we just do metric trees. So no, that doesn’t really work. You still need specific, let’s say explorative dashboards to figure out why a metric is actually looking a little bit wonky this month.
Shane: I think back in, in the balance scorecard days, there was this concept of simulation. , It was almost like a digital twin. And although that term wasn’t around then from memory, so you could actually go in and say, if I change this input metric by a certain amount what’s the flow through?
And from memory we were inferring a delay. like you said, number of prospects coming into the funnel, how long would it be before they create an account and go into a pay plan? How long would it be before that money turns up if it’s a 30 day window for the bill? And so you bake that into the model to a degree so you could simulate some changes, but, that was quite technical.
Timo: What I often do is I take a metric tree and I just break it down on a spreadsheet and then create something which we can call a growth model, where I then have the metric tree on the left defining all the different kind of rows, and then I can put the timeline on the right in there, and then I can just model what you just described.
I can do some forecasting. And for me, the interesting part is I see the mechanics of the business and then look, I can change some conversion values and can see, okay, what impact does it have when we actually increase this by 10% and so on. And then I see it how it plays down. So it’s a very basic and amateur way to do forecast simulation, but often quite enough for most of the stuff
Shane: And again, , that relationship’s an interesting one and you just made me think about it. So we’re doing a whole lot of work in what we call the context planes. It’s this idea of taking the context of our data and bringing it back into, for us it’s centralized.
And we think about it as four types of context we think about as business context. Actions, outcomes, glossaries descriptions, those kind of things. We think about it as a structural context, so physical schema data types and all those kind of things. We know about our data. We think about it as operational context.
So when was data loaded, when was it refreshed? What’s the quality score? Those kind of things. The last one we think about is agent context, the prompting, reinforcement, those kind of things. And so there’s a bunch of object types. We have, so we have, actions as an object type outcomes, an option type metrics as an object type a fact, a value as an object type.
Then the last part of it is the relationships across those object types. And the reason we’re working on this is this context map. Effectively, if you give it to an LLM it’s really good at using it to answer questions. So you can do what we call blast radius. If I change this bit of code.
If I change this object, if I change this metric, what’s impacted. And so thinking about metric trees, they’re giving us a relationship across those metrics, which is effectively describing the business model for the organization. And that relationship would be really valuable to an LL lm if you’re using an AI agent because you can say, if I touch this metric, you know, what the relationships are infer what’s gonna happen, and that actually might be a non-deterministic way of getting a simulation quicker than having to build really complex data models.
Timo: Yes. So I was experimenting a little bit with that. Let’s say in the easiest way, you just do a copy of of your metric tree and put it in and then say, okay, can you please run this in specific ways. I also did some experimentation with, let’s say, descriptive YAML format for metric trees was not really happy with that. So I definitely, so far I have abandoned the idea to do this. But no, you’re right. Also because when you put it in LMM, then for example, you can ask, okay, can we run a cohort simulation on top of that? So let’s assume we run this initiative in the next months where we think, okay, we will increase this kind of metric, and then we want to see how the impact will look like over the next month.
And what I often see. For example, when I have, let’s say full data set up for let’s say a startup. And then the metric tree is always you can use it in an l and m and then you can say, okay let’s brainstorm on this. So right now we have this problem here, or we want to dive deeper into retention.
It’s this is the whole picture, so can you help us to break it down? So can you give us different kind of versions of what, let’s say what would the level underneath look like? And still at the moment, this is my favorite LMM workflow or let’s say use case is really to use it as this massive brainstorming machine where you can say, look, okay, let’s look at this angle.
And I can say, oh, okay, let’s slightly change this kind of angle. If we bring this, and then because you have so many things, then. Mapped out to you so you can immediately say, oh yeah, this is the direction we should follow on. Let’s go deeper there.
So it definitely helps to not do crazy stuff because you give it a, let’s say, a form or structure
Shane: And then my natural reaction is, oh no, because LMS are non-deterministic, and so the simulations it’s gonna run is wrong, but we’ve just already said that, you Correlation of cause and effect across the metrics is hard to do if not impossible. So a non-deterministic engine is just behaving like a human going, this is what the patterns we see.
So actually using it to do those what if analysis, those simulations, those, how are those things related is just as valuable as humans doing it. I hadn’t thought about it that way.
Timo: and the Usually nowadays would, take the other approach and would write a Python program.
Often, when you ask these things, they say, yeah, let me write a quick script for you. Usually does that, and then you can just check.
Shane: Interesting space. I think , we’ve seen metric layers come out as bi semantic layers. And they’re not, from my view, they’re not making it right. They’re not really getting traction on the market. Maybe we’ll see a reinvigorated of the balanced scorecard products as metric three products.
‘cause there’s probably some value there. Alright so let’s just go back to basics, i’m rocking into an organization, or I’m an organization and I wanna start off this journey. We’ve talked about the map is the most important thing, the shared conversation. So we need to build that out over time.
And we’ve talked about doing it, step by step, don’t bore the ocean. We’ve talked about the fact that actually defining metrics are hard. . So each one of those you’re gonna think is, oh, it’s only MRR. How hard can that be? You’ll find out. So we know that each metric to define it and implement it actually is a lot harder than we normally think and takes time.
Where do you start? , So if we think about this idea of input and output metrics and metrics in the middle that we’ve got this idea of a heartbeat metric, the core metric for your organization, that actually is the one you want to look at the most, and that’s gonna be made up of a whole lot of input metrics, we know that. Where do you start? Do you tend to start at the input side? Do you tend to start at the output side? Do you tend to start in the middle? , How do you decide which metric to do first?
Timo: Huh, that’s a good question. I start on two areas. Area number one is I usually bring, let’s say, a set of people of this company into one room. That can explain, let’s say the whole customer journey or the whole customer flows in this company to me. I usually work with, let’s say startups, which are between seed and series A.
So therefore, the whole business model or the whole business processes are not super complex yet. , So it’s definitely possible in three hours to map the whole thing out. So I do a classic event storming session. So not classic event storming is a little bit broader and it does more things. So like I do a reduced version because I’m just interested in specific things. But. When you ask a company to explain how the customer’s basically going through the whole process, then you will identify these things. So you will identify, okay, what are the actually important points that they have to come to that in the end? Like it means success for you. So we will find this one place usually pretty quickly where we say, okay, this is a sweet spot. So that we definitely have to have a metric for that. We just have to see how we define it. For example, an active user is a North Star metrics in general. It’s it’s a definition process that takes longer because you come up with a hypothesis where you say, okay, people have to, okay, I’m just making this up, let’s say when you’re miro, so they have to create one new dashboard every month. They have to share at least two. And then they have to add at least 20 cards on a board or so, so then you would say they’re active. So this definition has to be fine tuned over time to see, okay, does it actually stick? This comes later. So I start with understanding the whole process. This is part number one, part number two, and this was something that I learned later. So I started very early on with these event storming maps because I use it already in other projects for other things. They’re really great to understand the process very quickly. The part that was always missing for me, that came later for me was. I sit down with the leadership team to understand strategy. I really want to understand where they want to go in the next six months. So what is the thing they want to improve which kind of direction they want to go, because from there I get the priorities, what you just asked.
Okay, where do we get started? So what do we have to build out first? I always try to bring it as close as possible to the strategy that the company right now is doing, because if I do it, I usually get a lot of more interest by management teams. By everyone I mean then by everyone else because they have to report towards this. I won’t do this, people would like. Yeah. It’s interesting, but it doesn’t really. Let’s say count into our current initiative. So if I fail to support a current push, strategic push into something, then usually I get a really low adoption rate. So this is like the second, I would say most essential part that I really try to understand, okay, what they’re trying to achieve in the next six months.
Shane: So once you’ve blueprinted out or, event stormed the kind of, I, I think of it as a blueprint, you’re creating a hypothesis of what the metric tree might look like when you’re finished. Then what you’re saying is you then look and talk to ‘em about their strategy to figure out where you should zoom in on you’re basically saying as part of that event, map that metric map.
And based on your strategy. I think we should do these ones first because they seem the most valuable, the ones that’ll get the most traction or have the largest conversation and adoption internally, which is the goal, is to get more people to understand it and use it and go, yeah, these are valuable.
Let’s do the rest of them.
Timo: It’s even slightly different. It’s not like just highlighting the areas. It’s often there’s a different version of a metric tree that I use. So you can take a metric and you can break it down, buy specific things. So this is the classic thing that you do in BI as well. So you have sales, you can break it down by something. You can do the same thing with the metric tree. So you can take a total metric and then you can break it down by some kind of segmentation, whatever it is. In this segmentation, I can try to incorporate the different kind of strategy. So I want to have, let’s say some, okay, let’s say, miro, I’m always using Miro because most of the people have used it before, so let’s say they push into AI supported workspaces. So then I’m trying to get up metrics that will highlight these, let’s say the adoption of AI supported workspaces versus non AI supported workspaces so that everyone on the first glance can immediately see, are we making progress with that or not? And so it’s really like creating a variant of the generic metric tree to say, okay, this is how it would be useful for the current strategy.
Shane: Okay so lemme play that one back because I just I want to make sure I get it. And is this idea of the buy statements of these metrics, I want this metric buy channel, before we started, we talked about the tools we use to create our content. And we talked about the fact that I use InCast for recording, I use Descrip for editing and I use Podbean for publishing.
And now what I’m seeing is a convergence. Yeah. I’m seeing each of those three products add the other features , that I do need into their product. And I was ranting about how that actually degrades the value of the one thing I use ‘em for. So let’s say that we’re zencaster and we wanna move into the real time streaming and video editing.
So what I’d be looking for is the metric of. Active, whatever, maybe active user, maybe active sessions. And then I wanna be saying, okay, how many people are moving from audio recording only to video recording? Because that’s a input metric. If they’re not doing that, they’re never gonna use our video editing feature.
And then how many of those move from video recording to video editing and then segmenting it maybe by region? And we say actually the US is the target market, so how many people? And by doing that, we’re effectively going, there’s a really small metric. Effectively, we’ve broken it down to its smallest part, but it’s there to support our strategy of getting more people using our video editing.
And so that’s an okay, that, that makes sense for me.
Timo: No, Is a perfect example. So exactly like that. And usually it makes it a lot easier than to talk to the people and people who find it very interesting. They want to see the results from that.
Shane: And you can get instant feedback. ‘cause somebody can say, actually yes, we are going after the video editing market. That’s one of our initiatives. ‘cause we all know that most organizations have 52 initiatives every quarter. But they can say, that’s not our most valuable initiative. This one over here actually is our most valuable and that’s what we should work on.
So when you are talking, you’re bringing a whole lot of patents to the table to support the metric trees. So you are talking about event storming. You’ve talked about jobs to be done. You’ve talked about domain driven design, you’ve talked about journey mapping.
There’s a whole lot of really valuable product and data and agile patents you’ve just talked about.
Timo: You can see. where I’m coming from. So it’s there’s a big product influence in this whole thing. Yeah.
Shane: They’re all valuable patterns that support what seems to be a simple pattern of metric trees, which is define a metric and say how they relate,
Who does the work then? Because if I talk about a data team, often I don’t see data teams applying product thinking or product patterns, they’re data teams. And so you get this idea of a data product manager now who kind of bridges it, but who is the type of person that would do this work given the level of patent understanding and experience that you actually need to make metric trees work to the best ability, not just defining a metric on a dashboard who do you see doing it?
Timo: That’s an excellent question. I don’t really see data product managers. Obviously because I don’t really see them happening so much. So from my experience when I talk to some people who use metric trees, I would say most of the times these are head of datas leading the data team.
Why is it good fit? Because they have a strategic role and some of these people struggle a little bit to , have a strategic role because they love data stuff. So it’s a tricky process to go from something which, was very operational to something which becomes more strategic. But let’s say the one or two people that I had long talks about were all heading data teams and they were. Completely concerned about, it’s okay, how can we do a better job to connect what actually the business is doing? They also let had a high frustration level, to be fair. So they define Northstar metrics without us, and so they, they do all these crazy things where we actually, as a data team should be involved, but there’s a reason why they don’t talk to us, so maybe we are not really well prepared for that.
And so this is where they invest their time. And I think for someone who’s leading a data team, I think all these things are good things to learn because they have a very strategic aspect to it. And You have the possibility to understand the business, which helps a lot.
Let’s say it’s the head of data, you are the. Chief sales officer for all the data work. So therefore, to have a good idea where your work makes most of the impact and gets you, let’s say, gets your team a lot of fame is definitely not bad to learn. So I would place it most likely there to not also create a different new role for that.
Shane: And it comes back to that value of the data team, value of the things they’re doing. And in my head, so within the information product canvas, we have this area called business questions, and the questions that we want to answer with that product. And from there I can infer metrics relatively easy.
And so therefore, if I think about it as a map, it goes, business question is supported by a metric. And an action is supported by a business question. If I answer that business question, what action are you gonna take? Now, when I think about metrics in a metric tree as more a business model, I’ve gotta think about where it fits in my, in the model in my head, because now I’m going, oh, actually, hold on. They’re not really a subset of a business question. They’re sitting in this patent map in a different place, and . I’m gonna need to think about that one. ‘cause that’s changed my thinking around metric trees, because
I just started off this conversation around it’s just a metric and yeah, I need a metrics layer rather than it is actually a simulation or a visualization of a business model.
Timo: Yeah. Yeah. Business process. So like both, It’s more in this direction.
Shane: yeah. That’s perfect. All right. So if people want to find you and find your writing and you run a course that teaches some or all of this stuff,
Timo: yeah, not yet. Okay, so a guest and I, so if, let’s say, if you look for Metric tree content, so you will come across, a small group of people who wrote about it. And so a guest is one, I’m another one. There are some others, Arby has written a lot of stuff about it. Ollie as well.
So we are working on a course but we want to do it right, so therefore this takes a bit of time. So we did already some iterations. We have some ideas now in general, if people want to read what I’m writing. So you can go to de odeo.com where I write my newsletter. We sometimes write about metric trees. Metrics, product analytics event data models is usually like what I write about. And sometimes I do this as well on LinkedIn. You can also just follow me there if you have a direct question. Or you can also just write me directly on LinkedIn. I usually try to answer them.
Shane: Excellent. I read your content. It’s great. You can tell when somebody knows your craft and then they spend the time to figure out how to write something that teaches it. ‘cause I dunno about you. I find I can write content quickly if I don’t worry about simplicity. But when I try and write something to explain a complex thing with simplicity, that takes me a long time to get it where I’m like, yep I think I’m happy.
And then testing it, give it to somebody and they can tell me what it actually means. That is a lot harder. And the way you write, I can read it and go, ah, actually I think I get it.
Timo: That’s good. No, like I just wrote a, I think this is the longest that I ever wrote, like an 8,000 word piece about the history and future of digital analytics. And I think this thing was brewing in my head for four months back and forth with different kind of variations. So like some sketches on a paper and then, ah, I’m not really happy.
No, this is the wrong direction. And so yeah these things take long until
Shane: so on that one , what we’re seeing now is we’re seeing generation of AI apps are incredibly fast and, 30, 40 years ago, we used to struggle with one system sitting on, mainframe or a, or whatever and getting the data. And then we moved to enterprise resource planning and CRMs.
So we always ended up with five to 10, and then we moved to software as a service. So we ended up with 15 to a hundred to a thousand systems. Now, with the ability to spawn up, an app in a heartbeat, we are gonna end up with a problem of, a thousand, 10,000 systems that capture data for an organization that allows that to happen, that’s gonna change the way we do product analytics, isn’t it?
Or is it not? Do you think that the techniques and patterns and technologies we have today are gonna be able to handle this idea that there are 20,001 shot apps in an organization that are being built for a very small use case for a very small set of personas.
Timo: That’s a very interesting question. I would say in general, not so for example, what really took me, some time to get to this point, others came to this point as well. Unfortunately it’s not so much teach and product analytics that in the end, the tricky thing in product analytics, where do you find an high abstraction layer that still explains the product enough, but it’s simple enough that you can, basically do some calculation and good stuff around it.
So , what really works well is the user state model, which is in the end a gross model. So you say, okay, we have a new user. You new user can become activated. It become active, it can become at risk. When it stop being active, then it can be dormant when basically nothing happens anymore, and then it can be resurrect and whatever.
So you can basically create a loop where people can move through different kind of states. If you take this kind of model, then you can have 400 things under it, as long as you can map the things that are happening in this. To one account. This I is always the tricky thing in product analytics. So as long as you can do this, you can still then abstract it on this high level. You can still say, okay, people do different things. Or , let’s say we capture different things, what the people are doing and let’s say these 100 internal tools that we are using, as long as we can bubble them up into one place where we can say, okay, look, when people show these kind of activities, we flag them as an active user. I would say the system still would work. I would say this is the only escape. If you go the, , classic product analytics approach where you try to track everything and then try to figure something out, obviously this is something you cannot win. You really have to go on a very high abstraction layer. I think the tricky thing is really how you do identity stitching. So let’s say the basic function of making marketing and product analytics work is you have to combine all these different kind of things. So if you cannot combine all the different kind of signals happening somewhere then obviously you cannot analyze them for this one account.
So then you basically have 20 phantom accounts that are actually one, but you have no idea that they are. And this I think, can be really an issue, especially when these tools can pop up everywhere and the company, you don’t have a concept to make sure, oh at least we should identify the people in similar ways everywhere.
Shane: Actually, you’re just giving me a. Visual map in my head of how you put all this together potentially. So effectively what I think you are saying is, when we have 10,000 apps that are one shot apps, the first thing we have to do is make sure we’ve defined the core concepts of our organization.
So concept of a user. Yeah. And that’s important. We know that anything to do with a user is important to our business. So that concept needs to be defined and held. And then if your application is touching a user, touching that concept, then you need to be aware of that. So you are touching the user concept, and that’s an important concept for us.
And then we have the metrics that are a form of statement around active user but also state of user, so there’s a state flow. So your application actually has to be able to do whatever it needs to do to tell us when there’s a state change for that. So we can define active or inactive.
Timo: the interesting thing is usually I like to model the state changes, so I don’t want to have the applications to refine it because we will play around with this. So we will have different definitions over time. We might even break it down. We might have six definition of an active user. This is a great stuff that you can do in a data model, in a data warehouse.
I usually tend not to have it already on the application layer. I just need to track all the things that are happening. This is why events are nice. ‘cause events I can use to derive a state change.
Shane: But to do that from a government point of view when you’ve got 10,000 apps, is you’re gonna have to say to them, in your app, when a user changes to this state, you need to push it out as an event that I can see.
That has to be a governed thing, is you have to actually do these events.
‘cause these are the core events, if you don’t do this, your app is not valuable. And then how do we know it’s not valuable? What we can do is show them the metric tree and say, if you’re not telling us that you are changing the state of a user, then this metric won’t move. And if this metric doesn’t move, then these other metrics don’t move.
And they’re our core North star metrics. Internal metrics, sorry, our estar metrics see I got it wrong then is like our internal metrics are North Star. No they’re not. North star metrics are about customer. So they’re our internal metrics, but they’re the important ones. So if you can’t tell us you’re improving that metric, then why the hell are you building that app?
And that it’s A combination of all those patterns and defining them early so that all those AI slop applications actually fit into this governance
Timo: I definitely have to make a case for AI slop applications. So I privately love to build them. Because now, for the first time, I’m a former product person. I still am a product person at heart, and my biggest problem was always. I need to see things that I can make decisions. So you can create a wire frame, sure. But it’s not really to have an app in front of you. So now you really have the possibility you can go down three ways. I would say, okay, I would do it like this. You can compare it and you can say, yeah, no, this makes totally more sense. So I think it’s really great what’s happening, but you’re completely right.
So it will create an interesting complexity for us.
Shane: Oh, we’ve seen this before and democratization is brilliant. The ability to put the tools in the hand of people that aren’t professionals in their art is great. , That is massively variable. We’ve seen it time and time again, but we’ve also seen the impact when we don’t.
Bring in the principles, policies and patterns that are useful. DBT allowed a lot of people to write transformation code, which is great. It removed the bottleneck of centralized data engineers who were never allowed to do it fast enough, but what we ended up with is 5,000 DBT, and I’m using air quotes here, models, and we lost the definition of a data model, of a conceptual model of those kind of things because we didn’t apply the policies, patterns and principles that were valuable. So I can just see with the ability for democratization of building apps, which is great, we are gonna have the same problem and for me, this idea of events defining them conceptual model and a metrics tree could be the things that we use to create those principles, policies, and patterns
Timo: that’s true. I also never thought about this in this way. I think it totally makes sense. No, you’re completely right. I’m a big believer in Democrats on one side, but on the other side, have a really good foundational concept that will tell you, where you define specific metrics that will tell you if the thing still work. So for DBT, my classic metric is how long does it take a new member of your team to understand the model and make a production commit. If you have 5,000 models, that’s will take eight.
Shane: Where, whereas my metric is what was the original time from a stakeholder saying they had a problem and being served with something that solved it with data and you’ve moved to a team of, 10 new analytics engineers and DBT has that time come down. cause if it hasn’t that’s great.
Busy work. Thank you for hiring more people and being really busy. And the other one that I often talk about when I run my course is the clock starts when the stakeholder says they have a problem. Not when that problem hits the Jira queue. Prioritize for the data team because if that takes three months, actually the stakeholders already said it’s three months late.
And it’s not the data team’s fault because they’re not allowed to work into the hits the team. But if we think about nodes and links and metric tum system maps now we just wanna focus on the prioritization process because that’s where it’s broken. That’s three months. And if the team take a day, it’s still three months in a day as far as the stakeholders.
So same kind of patents as you, eventually some form of storming, some form of jobs to be done, some form of. Domain driven, bucketing, some form of journey mapping. We can apply that to the way teams work as well. It’s the same set of patterns and they have value. Yeah, end of rent. But , I can see that metrics tree and that event definitions of those core events has been really valuable in the, in, in the democratization we’re moving to.
Excellent. All right, so at the beginning , you said, oh, I’m not sure we can talk about metric trees for an hour. And I said we’ll talk about lots of stuff as we did. . Hey, look, thank you for coming on the show. It’s been awesome. I’ve learned lots and I hope everybody has a simply magical day.
«oo»
Stakeholder - “Thats not what I wanted!”
Data Team - “But thats what you asked for!”
Struggling to gather data requirements and constantly hearing the conversation above?
Want to learn how to capture data and information requirements in a repeatable way so stakeholders love them and data teams can build from them, by using the Information Product Canvas.
Have I got the book for you!
Start your journey to a new Agile Data Way of Working.