Metrics without a model: understanding what actually drives the money

The weekly meeting looks professional: conversion, retention, active users, average order value. Charts update without delay. Yet when it is time to decide next week’s work, the discussion goes back to opinions.

Often the report answers “what changed” while the team needs “what should we work on”. Between them sits a testable explanation: which user actions create the outcome, where it is lost and how it connects to money.

Choose an outcome you can observe

Imagine a shift-planning service for small chains. Registration does not demonstrate value. Neither does a draft schedule: a manager may simply have opened a template. A more meaningful event is a published schedule with required shifts covered and staff notified of their assignments.

Even that event starts as an activation candidate. Use conversations and observation to check whether the manager really got a result. Then examine its connection to continued product use. Association helps select a metric but does not establish causation on its own.

AJTBD uses the concept of a critical chain of jobs: necessary tasks without which the final outcome is impossible. Mapping that chain to product events is useful for diagnosis. A click funnel only shows the part you have learned to measure.

One funnel drop, several explanations

Take a hypothetical cohort of 100 new managers. In seven days, 72 start a schedule, 49 publish it and 31 return for the next planning cycle. Every number refers to the same people; follow-up is long enough for everyone’s next cycle to occur.

Twenty-three people were lost between starting and publishing: 23 percentage points of the initial cohort, or about 32% of those who started. Do not conflate these in a presentation; they answer different questions.

The numbers do not yet explain why people stopped. The manager may not understand the interface, may find that staff constraints are unsupported, may be waiting for a colleague’s availability list, or may already have solved the task in a spreadsheet and opened the app out of curiosity.

The same chart can lead to four different decisions. That is why one conversion percentage should not automatically trigger a redesign.

Connect the change to economics

Suppose the product receives 400 qualified sign-ups a month. The team expects a better first workflow to raise the paying share by five percentage points: 20 additional payers. If each contributes ₽3,000 in the first paid month, the model projects another ₽60,000 for that month.

This is a scenario, not an observed effect. Deduct extra costs, check retention and price the change itself. The point is a transparent chain of assumptions: if the payment uplift fails to happen, the financial result will not appear either.

Such a model lets you compare activation improvements with other moves: changing a package, an acquisition channel or service cost. The decision no longer depends on which chart shows the most alarming drop.

Agree on what will count as evidence

Before release, record the primary metric, observation window and guardrails: more published schedules, for example, without more incorrect assignments or support load. When changes affect teamwork, organisation-level assignment is often more sensible so people in one workspace do not receive conflicting variants.

Sample size depends on the baseline, expected effect and tolerated error. A week is not sufficient just because the sprint ended. With low traffic, use observation and a limited pilot to refine the hypothesis, explicitly noting that a precise quantitative effect is not established yet.

After the test, three decisions are valid: expand the change, rework it or stop. A useful report ends with one of them and explains which evidence supports that next step.