Three weeks ago Anthropic launched Claude Mythos, a frontier model whose cybersecurity capabilities — discovering vulnerabilities, building working exploits, chaining them into multi-step attacks — sit materially above prior frontier models. The UK AI Security Institute (AISI) confirmed the step-up. Mozilla’s Firefox 150 release shipped engineering work at a scale Mozilla had not previously published in a single advisory. The defensive contribution on the slice of the cybersecurity surface that source-code AI auditing addresses is real.

The framing layered on top of that contribution has run ahead of it. Mozilla’s blog headlines 271 vulnerabilities for Firefox 150 while explicitly crediting Claude on three of forty-one Common Vulnerabilities and Exposures (CVE) entries. OpenAI’s “over 3,000 critical and high fixed vulnerabilities” claim from April 14 escalated to “11,353 critical and high-severity issues” by late April — a four-times jump in two weeks, with the counting unit quietly shifting from fixed to identified. These are competitive scoreboards, not engineering disclosures. The capability is real; the scoreboards are real; they are not the same thing.

The topic is technically dense and structurally tied to multiple strategic surfaces — defensive economics, supply-chain integrity, jurisdictional access, sovereign capability — none of which resolves to a single narrative. Announcements read as engineering disclosures but operate as marketing claims; separating the two requires working through each surface independently. I wrote a ~40-A4-page primary-source-anchored monograph on that basis. This post is the synthesis; the article is the work behind it, downloadable at the end of this summary.

The synthesis is a three-thread judgment.

First, Mythos is the industrialization of known-class vulnerability discovery, not a new category of capability. Mozilla itself reports that Mythos has not surfaced bugs an elite human researcher could not in principle have found. The change is in throughput and delivery mechanism — elite-style reasoning, machine-scaled, available with relatively little bespoke scaffolding. That is significant, certainly welcome, but it is not a paradigm shift.

Second, supply chain is the dominant blind spot in any defensive posture centered on source-code AI auditing. Eleven high-profile supply-chain compromises were examined, from SolarWinds (2020) through the March 2026 Trivy / aquasecurity case. None would have been prevented by AI-assisted review of legitimate downstream consumer code, because the malicious change typically did not live in the source.

The volume signal aligns with the case record. Sonatype’s 2026 State of the Software Supply Chain records 454,648 new malicious open-source packages discovered in 2025 alone — a 75 percent year-over-year rise.
In contrast, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerabilities catalog, the public registry of confirmed in-the-wild CVE exploitation, grew by 31 percent year-over-year in the same window.

The supply-chain attacker surface is expanding at more than twice the rate of the catalogued CVE-exploitation surface — a structural comparison rather than a head-to-head one, since the two metrics measure different slices. Per-incident blast radius reinforces the point: a single supply-chain compromise can affect tens or hundreds of thousands of downstream consumers (SolarWinds: 18,000; tj-actions: ~23,000 GitHub repositories; IndonesianFoods: 169,538 packages; Polyfill.io: 384,773 hosts). A memory-safety CVE typically affects users of the patched product - though the impact can be large with operating systems, browsers, or media players.

Third, access to Mythos-class capability is structured along jurisdictional lines that produce competition-distortion effects independent of technical capability. The named Glasswing partner roster is American — ten US-headquartered firms plus the Linux Foundation. The European Commission is in active discussions but does not have access. The UK does, probably along the same intelligence-sharing geography that has governed transatlantic security cooperation for decades, or because AISI has no equivalent in other EU countries in terms of technical capabilities.

A coherent European chorus has formed on the public record. France’s CIANum, Germany’s BSI, and the Bundesbank have each framed the asymmetry as a sovereignty or competitive-distortion concern. The argument is no longer Franco-French.

For European software companies in particular, the procurement-defensible question changes. A buyer evaluating two functionally equivalent network or security appliances in 2026 now has a defensible reason to prefer the vendor whose product was Glasswing-audited — not because of any belief about whether Mythos found anything load-bearing, but because one vendor was audited at frontier-AI capability and the other was not.

For decision-makers, the three threads translate into specific moves.

If you are a software vendor CEO whose competitive segments overlap with the named Glasswing roster: audit the procurement-defensibility exposure on a 12-24 month horizon. Position around quality, operational discipline, and externally-verifiable build-pipeline integrity rather than around frontier-AI access equivalence — the access asymmetry is unlikely to close on your timeline.

If you are a CEO, CPO, CTO or CISO at an organization outside the Glasswing roster: deploy AI-assisted code review on the highest-risk components first (authentication, session management, payment flows, file upload, deserialization paths) using whichever tier of frontier-model access is commercially available; instrument the triage pipeline before scaling discovery; pin all CI/CD action references to immutable commit crypted signatures (the Trivy and tj-actions lessons); audit dependency-update workflows for trusted-publishing posture.

If you are a board director or PE partner: the access-asymmetry structure is a 12-24 month consideration for 2026 software-sector strategy and acquisitions. Treat it the same way you treat any regulatory or supply-chain dependency that is structurally asymmetric across jurisdictions — a category of risk to factor into investment theses, not a precise pricing model the public record can yet support.

Across all three: extend governance attention beyond “do we have AI source-code auditing” to also include “do we have build-pipeline integrity controls deployed, default-on, enforced, and externally verifiable” — the controls are complements, not alternatives. Invest in the sovereign and operational levers that reduce jurisdictional access exposure while the asymmetry window is open — its resolution, whether through replication or entrenchment, is a 12-24 month question.

The cybersecurity contest has entered a new phase. Frontier-AI cyber capability is real, dual-use, and proliferating on a vendor-forecast timeline measured in twelve to twenty-four months. The right response is neither dramatic-shift rhetoric nor business-as-usual complacency. It is operational discipline applied to the three threads — capability calibration, supply-chain integrity, jurisdictional posture — at once. None alone is sufficient.

To be clear: invest in cybersecurity, now, and substantially. No vendor can afford a reputation hit from insufficient cybersecurity. As Thorbecke put it: ‘Trust arrives on foot and leaves on horseback’ (Vertrouwen komt te voet en gaat te paard, Johan Thorbecke, 1798–1872).

The judgment compresses to one operational sentence:

Mythos changes the economics of finding. It does not, by itself, change the economics of fixing, shipping, proving, governing, or accessing defensive capability.

The full monograph — primary-source-anchored, with a reference list, definitions, cross-jurisdictional press coverage, and a CO2-footprint calculation — is downloadable below. Four reading paths inside the PDF: end-to-end, skim for senior tech executives, technical-cyber, decision-maker.

Use the PDF to print; the ePub format may be easier on a tablet.

Claude Mythos Industrialization Not Revolution

1.55MB ∙ PDF file

Download

Download

Claude Mythos Industrialization Not Revolution

198KB ∙ EPUB file

Download

Download

This analysis is free and will stay that way — access to evidence-grounded security thinking should not depend on a subscription fee. A subscription supports the work.

By Mathias Przybylowicz, Founder & Principal, Boxing Day Consulting · May 2026 Boxing Day Consulting — helps software companies overcome strategic and technical obstacles to achieve profitable growth.

I once got cornered at a management offsite. We were around 150 engineers at the time. Revenue was growing. Headcount had substantially grown in a few years. On paper, everything looked decent, despite the occasional hiccup.

During a plenary session, my boss turned to me in front of the whole leadership team:

“We had a customer NPS review recently, and you all know it was a disappointment. Why is it so slow product-side? I’m hearing that small improvements take ages. Stability is also a challenge.”

There were real issues -- we were addressing them. But the Director of Ops seized the moment and listed recent incidents. The CPO, caught off guard, stayed silent – or cautiously chose his battle 😊. The VP of Sales added customer complaints and a recently missed date (a release I had vetoed for quality reasons). Within five minutes, it became a public diagnosis: Engineering was the bottleneck.

I was unprepared.

It took months to untangle what was really happening. The problem was not tools, skills, not even resources. It was scale – both in the tech stacks and in the organization. We had crossed a threshold without redesigning how we worked.

Something changes around one hundred engineers.
The number itself is not scientific. It is a useful threshold. Around that size, you no longer know everyone, and you likely have more than one engineering / product / ops location. Informal trust weakens. Shared context fragments. Coordination stops being a social reflex and becomes a design problem.

Below that line, coordination is mostly social. Above it, it becomes structural. What used to work through trust and proximity now requires deliberate design. If you don’t redesign the system, the system redesigns you — usually painfully.

If you come from a strong Agile background, this may sound like a problem that should not exist. In principle, cross-functional teams aligned to value should remove most of these handoffs. In practice, too few organizations at this size actually operate that way.

Whether you call it that or not, you are running a system that turns market needs into running software. In well-designed organizations, much of this happens within the same teams. In most, it doesn’t.

When this “factory” is small, you can run it informally. At scale, informal mechanisms collapse under their own success. AI is starting to act as a multiplier. It makes good systems better — and broken ones fail faster. In several organizations, this already shows up as a paradox: more code produced, but less reliable delivery — and no clear understanding of why.

The underlying design issues do not disappear. They become visible sooner.

And this is where many software organizations hit their socio-technical cliff: structure, code, market pressure, and culture all aligned in the wrong direction.

How the Breakdown Shows Up in Real Life

You don’t need a dashboard to feel that something is wrong at 120 engineers. You feel it in meetings. In Slack threads. In the tone of customer calls.

Three patterns appear again and again.

1. Delivery slows while work piles up

Everyone is busy. Yet simple changes take weeks.

A two-line configuration adjustment touches five teams. An unclear amount of manual regression needs to run upon changes.
Consequence: a minor UI quality-of-life tweak for customers waits behind three parallel initiatives because “we can’t release too often.” Batch sizes grow. Risk grows with them. Deployment frequency drops. Lead time climbs.

Estimates start to feel fictional. Roadmaps get “re-baselined.” Sprints are descope-festivals.

Operations (Cloud Ops, Support, Professional Services – up to Presales) feels the strain first. Incident volume increases. Resolution times stretch. Important accounts notice long before the management team does.

Velocity doesn’t collapse overnight. It erodes. And that erosion is not just technical. It shows up in missed revenue, delayed deals, growing cost to serve, and margin pressure that no efficiency program seems to fix.

And the more pressure you apply, the worse it gets.

2. Structure adds friction instead of clarity

At this scale, silos rarely look like silos on the org chart. They look logical.

Feature squad.
API team.
Platform.
Security.
Data.
Ops.

Each makes sense. Together, they create dependency chains.

None of this is how Agile is supposed to work. In theory, teams are cross-functional and aligned to value, precisely to avoid these handoffs.
In practice though, once organizations grow, specialization creeps in — often for good reasons — and the seams reappear.

One feature crosses four backlogs and three approval steps. Architecture mirrors reporting lines more than customer value streams. Security reviews happen as manual gates outside delivery pipelines. After a few incidents or audit scares, the instinct is predictable: add process.

More steering committees.
More templates.
More approval layers.

Ceremonies multiply. Flow does not.

Most of these organizations will tell you they are Agile — and they are, in terms of rituals. But structurally, they still operate as a chain of specialized teams.

This is where agile theatre creeps in: busy calendars, unclear impact.

3. Ownership blurs and energy drops

At 40 engineers, misalignment is a nuisance. At 120, it becomes a cost center.

Ownership diffuses. At that point, reducing headcount may look like a solution. In practice, it often makes the system more fragile — because the underlying coupling and dependencies remain. It becomes easier to assume someone else will fix it. Product owners are turning into story machines rather than problem owners. Engineers feel they are moving tickets, not solving customer issues.

There is always one Slack channel where everything eventually lands — and a handful of “heroes” who silently hold the system together.

They become bottlenecks by virtue of competence.

Over time, strong engineers and solid PMs leave with the same sentence: too much chaos or too much bureaucracy, not enough impact.

By the time the management team sees declining predictability, the teams have been feeling it for months.

Why It Really Breaks

The symptoms look messy. The causes are surprisingly regular.

In most cases I’ve seen, four forces reinforce each other.

1. Growth without an operating model

Teams are added. Managers are promoted. New roles appear: SRE, platform, product ops... But the underlying decision model remains implicit.

Decisions still funnel through historical leaders. Autonomy is celebrated in town halls and quietly constrained in weekly governance. Teams are accountable for output, not outcomes. Matrix hell creeps in: “you’re not my boss”.

When scale increases, ambiguity compounds.

2. A technical system not built for parallelism

Codebases built for two teams now host ten. Boundaries are unclear. Test suites are fragile. Environments are slow. Internal platform work has been postponed for “after this quarter.”

When you finally need speed and reliability, every change touches too many components. This is not about bad engineering. It is about accumulated architectural gravity.

And gravity always wins.

3. External pressure lands at the wrong moment

Sales wins larger deals with fixed dates. The Management Team pushes for margin discipline. Regulation tightens. Security audits multiply.

From a business standpoint, all of this is understandable. But it lands on a system that is already fragile. At that point, pushing harder does not increase output. It increases failure rates — and hides the real cost behind rework, churn, and lost credibility.

Leaders often feel caught in a paradox: invest in platform and structure to improve long-term performance — or protect short-term numbers. Both pressures are legitimate. The tension is real.

4. Culture doesn’t scale automatically

Oral alignment works at 30 people. It fails at 120.

If written decisions, shared metrics, and explicit trade-offs don’t replace tribal knowledge: confusion grows. Instead, metrics reward silo outcomes rather than end-to-end flow, and what should be a shared system fragments into local optimizations ... or fiefdoms.

At that point, “more autonomy” often turns into teams optimizing for themselves, while “more process” turns into additional coordination overhead — both adverse to the original intent.

Psychological safety erodes quietly. Concerns about technical debt or unrealistic roadmaps stop being voiced.

This is usually not dramatic. It is gradual.

Then one day, at an offsite, it explodes in public.

I have seen variations of this pattern across multiple organizations and contexts — the details change, the underlying mechanics do not.
Refer to previous articles to get further insight, for example:

• The CEO-CT/PO divide

• The “Software Engineering, what could go wrong?” series (link to part 1)

• The “Software Delivery Performance” series (link to part 1)

A Practical Way Through the Crisis

There is no single way, and no need for a big-bang reorganization.

In fact, large-scale reorgs under stress often make things worse. Framework-first transformations rarely fix structural issues. In some cases, they make things worse — by locking existing dysfunction into the operating model instead of removing it.

This is where organizations often reach for large-scale frameworks — SAFe or similar — as a way to regain control. There are useful ideas in them: prioritization mechanisms like WSJF, clearer cadences, shared vocabulary.

The problem is not the tools. It is how they are applied. In many cases, the framework is deployed as a whole, regardless of context, maturity, or actual constraints. What was meant to reduce friction ends up adding layers: more roles, more ceremonies, more coordination overhead.

The result is predictable: frustration increases, ownership decreases, and productivity drops — while some feel they are “doing Agile properly”, and others feel a bureaucratic monster has spawned and miss the “good old time”.

You see the same pattern with other reflexes: adding layers of management, multiplying coordination roles, or tightening approval loops. They create an impression of control, but rarely improve flow.

AI can help with that upgrade — faster feedback loops, better tooling, partial automation of repetitive work. But it only pays off once the system is coherent enough to absorb the gains. Otherwise, it just accelerates the existing bottlenecks.

A better approach is less visible, less heroic, and more gradual. It is also uncomfortable. Some initiatives will slow down. Some decisions will be challenged. Some people will opt out — including some of your strongest individual contributors or managers.

Think system upgrade, not revolution.

1. Stabilize and see the system

Start by slowing down — selectively. Not everything at once, but the parts of the system that create the most instability and rework.

Put Product, Engineering, and Ops around the same small set of metrics: lead time from idea to production, deployment frequency, incident rate, mean time to recovery, customer impact.

One shared picture.

In overloaded areas, cap work in progress. Say no to new commitments temporarily. This is not a comfortable move. It means trading short-term revenue or scope for restoring delivery capability. The alternative is to keep selling into a system that cannot deliver — which is usually more expensive, just less visible. At that point, you are not accelerating growth. You are front-loading failure.
You cannot accelerate a traffic jam.

The goal is not speed. It is stability. You cannot maximize speed, flexibility, and reliability at the same time in a system under stress. Something has to give, at least temporarily.

2. Diagnose constraints, not people

Follow a few real changes end-to-end. Where do they wait? Where do they bounce? How many teams are involved?

Name the patterns: monolith bottlenecks, hero traps, bureaucracy loops. Then pick one or two constraints that actually limit delivery today — typically where work waits the longest or fails the most. Fixing everything at once is how most transformations stall.

Treat these patterns as system issues, not individual failures. That does not remove accountability. It clarifies where leadership has allowed the system to drift — and where it must be corrected. Most of the time, the system is not broken by accident. It is the result of a series of reasonable decisions that were never revisited.

And be explicit about what does not help. Which meetings exist because “that’s how Scrum works”? Which gates survive from past incidents but no longer add value?

Remove one old rule for every new one you add. In fact, aggressively reduce their number to what’s really needed to produce outcomes.

3. Align on clear rails and real autonomy

Clarify responsibilities at the top. In practice, this is rarely a clean exercise. It means surfacing tensions that were previously implicit — between Product and Engineering, between Sales commitments and delivery capacity, often starting within the leadership team itself.

The CEO defines where to play and the financial envelope.
The CTO or CTPO designs and runs the software factory.
Product owns the problem space and portfolio logic.
Ops owns reliability and cost to serve.

Shared trade-offs are explicit, not implicit.

Then define a small number of hard rails: production access rules, incident management standards, security integration in pipelines, a handful of company-wide metrics.

Inside those rails, teams choose how they work.

More structure to protect autonomy — not to suffocate it. The goal is not to manage handoffs better. It is to reduce them over time.

4. Upgrade deliberately

Align teams with value streams where possible. Wherever possible, move towards teams that can own a problem end-to-end — from idea to production and support. The closer you get to that model, the less coordination overhead you carry.
Invest in platform capabilities that reduce cognitive load: faster pipelines, reliable environments, observability that engineers actually use.

In many cases, the first wins are not architectural rewrites but removing a few high-friction points — slow builds, unstable test suites, brittle environments — that block every team.

Move security and architecture checks into automation rather than manual approvals. Shift to smaller batches. Protect focus time. Allocate explicit capacity for technical debt and hardening work.

On the product side, rationalize the portfolio. Stop low-ROI initiatives. Restore direct customer contact for engineers and PMs.

When engineers hear customers describe real pain, prioritization debates change tone.

None of this should roll out as a transformation program with a launch event. You cannot fix this without temporarily trading off something — speed, scope, or short-term predictability. Start with pilots. Adjust. Expand. This is not a smooth process. Some teams will move faster than others. Some changes will fail. You will create new friction while removing old ones.

Expect resistance. Adjust again.

Scale as a Design Choice

Crossing 100 engineers is not a linear step. It is a phase change. Many organizations believe they are already structured to handle it. Few actually are.

If you respond with more heroics, you get chaos.
If you respond with heavy control, you get bureaucracy.

Both feel decisive. Neither solves the underlying system design problem.

Most companies think they are scaling headcount. In reality, they are scaling complexity, and most of their fixes make that complexity worse. Not because the ideas are wrong, but because they are applied without changing how decisions are made.
And unless that complexity is managed deliberately, it converts directly into cost and lost growth.

Scale is a design choice.

You decide how Product, Engineering, and Ops interact.
You decide where decisions live.
You decide which processes are non-negotiable and which freedoms are protected.
You decide what success means beyond output.

I have seen organizations cross this threshold painfully and others deliberately. The difference is rarely talent. It is whether leaders accept that the factory must be redesigned — not defended.

When that redesign happens, having 100 engineers becomes an asset.

Until then, it becomes a multiplier of friction.

About This Series — A Practical Guide to Reading Engineering Performance

Software engineering carries a unique tension: it’s a major investment, it shapes customer experience, and it determines how fast a company can learn or react. Yet for most CEOs, COOs, CFOs, and PE investors, engineering remains oddly opaque. Activity is visible, but impact is not. Conversations drift between too-technical explanations and too-high-level summaries.

This series closes that gap.

It brings together field experience and established research (DORA, SPACE, DevOps practices) to offer a clear, business-oriented way to read engineering performance.
It answers practical questions such as:

• Which metrics actually help leaders understand what’s happening?

• How do we avoid metric theatre, gaming, or over-interpretation?

• How do engineering signals map to revenue, margin, NPS, and risk?

• What should teams measure to stay predictable without burning out?

• How should cross-functional accountability work?

• What should buyers or investors ask during due diligence or renewal cycles?

Across three parts, the series builds a structured, audience-aware model:

• Part 1 – The Foundations & Team Layer:
  Why metric conversations break down, how to create a shared language, and the team-level signals that drive sustainable, predictable delivery.

• Part 2 – The Executive Layer:
  How engineering connects to revenue, margin, customer outcomes, and strategic priorities — and how leaders should interpret this information.

• Part 3 (this part) – The Collaboration & Buyer Layer:
  How engineering, product, finance, CS, and sales align around metrics; how to read reliability and operational maturity; and how buyers (customers, partners, PE) assess the organization.

This is not a theoretical framework.
It is a practical, tested approach to understanding engineering performance in real companies, under real constraints, with real business outcomes at stake.

If you haven’t read Parts 1 and 2, here’s what you need to know:

From Part 1:

• Engineering needs a shared metric language.

• Team-level signals (flow, stability, focus, collaboration) provide the foundation.

• These signals support predictability and sustainable execution.

From Part 2:

• Executive metrics must connect engineering to revenue, margin, NPS, and risk.

• Technical signals only make sense to the business when framed through value creation.

• Attribution requires careful framing, not overclaiming.

• The right metrics depend on strategy and require cross-functional inputs.

Part 3 pulls these threads together:
How to operate this model across functions, how to use it in real conversations, and how buyers — including customers and PE investors — should interpret engineering performance.

PART 3 – TL;DR

1. Cross-functional collaboration is a prerequisite

Engineering cannot compute executive KPIs alone.
Product, Finance, SalesOps, and Customer Success all provide data needed to form a complete picture.

2. Data Readiness Levels matter

Not all metrics are equally mature.
Acknowledge when data is approximate, and create a path to improve reliability before turning metrics into targets.

3. Change management matters more than the metrics

Teams need time to learn how to interpret signals.
Executives need consistency.
Rotating metrics or changing formulas creates noise and mistrust.

4. Stakeholder alignment requires clarity of incentives

• Sales optimizes bookings.

• Customer Success optimizes retention.

• Finance optimizes margin.

• Engineering optimizes flow and risk reduction.

A good metric model shows how these incentives interlock rather than compete.

5. Listening mechanisms make metrics real

• “What changed this month?” notes

• Quarterly listening posts

• Retro excerpts

• Thematic summaries of friction and progress
  Qualitative signals give meaning to quantitative ones.

6. Buyers need a structured set of questions

PE investors, integration partners, and enterprise customers can use specific indicators to assess engineering maturity:

• Reliability and SLO attainment

• Incident patterns and recovery speed

• Change failure rate

• Release cadence

• Security posture

• Feature adoption and time-to-value

7. Metrics are a language, not a verdict

Used properly, they enable trust and alignment.
Used poorly, they create pressure and distort behavior.

VII. Building the Bridge: Collaborating Across Business Functions

Many executive metrics related to Engineering Performance need input from Sales, Product, or Finance. Don’t stall, structure the collaboration.

VII.A Baseline Inputs Engineering Should Expect

Finance: recognized revenue by product/module.
SalesOps: CRM tags for product influence in deals.
Customer Success: NPS feedback categorized by product/feature.

VII.B What Engineering Can Compute or Maintain Internally

Infra/Support Cost per User; Cost per Adopted Feature; Time‑to‑Adoption; % time on strategic initiatives (epic/label tracking).

VII.C What Product/Finance/Sales Should Track Anyway

Product: feature usage/activation.
Finance: per‑product margin and cost allocations.
Sales: feature‑request ↔ closed‑won correlation.

VII.D Data Readiness Levels (DRL) & Accountability

Principle: never propose a metric you can’t reliably measure. Each metric must have both an Accountable owner and a Producing unit (e.g., Finance produces margin metrics; Engineering produces lead‑time). Otherwise, expect disputes and low trust.

VII.E Change Mechanism (Teams & Exec)

• Review cadence: monthly for exec scorecards, per‑sprint for teams.

• Proposals: anyone can propose to add / retire / refine a metric with a written rationale and owner.

• Criteria: keep 5–7 signals and ≤3 targets, also useful as Key Results (as in: OKRs); add only if it changes decisions.

• Sunset policy (with caution): retire only if flawed, no longer aligned to goals, or superseded; avoid churn so people learn to interpret signals.

VII.F Practical Rollout Plan

1. Pilot in one product area; 2 sprints of shadow reporting.

2. Train leads on interpretation; publish definitions and owners. Manage change actively, explain the Why.

3. Agree thresholds as ranges, not single points.

4. After 8 weeks, adjust; then scale.

VII.G Stakeholder Alignment Playbook (Overcoming Resistance)

• Incentive map: list what each function optimizes (Sales: bookings; CS: renewals; Finance: margin), then state how each metric helps their aim.

• Single source of truth: define where each metric lives (e.g., warehouse/BI) and who signs off monthly.

• RACI per metric: Responsible (producer), Accountable (owner), Consulted (cross‑functions), Informed (audience).

• Escalation path: unresolved disputes go to CT/PO+CFO for arbitration with evidence.

VII.H Qualitative Balance (Listening Mechanisms)

• Monthly “what changed” notes: each team logs 3 bullets: decisions taken, trade‑offs, surprises; attach to dashboards.

• Quarterly listening posts: 45‑minute small‑group interviews across roles; tag themes to metrics.

• Retrospective excerpts: capture learning, not just numbers; share highlights in QBR or MT meeting.

VIII. A Word from the Skeptics – and Why They’re Not Wrong

A fictional composite of Kent Beck, Allen Holub, and XP veterans.

I agree with the spirit. Still, CTOs must explain performance to boards that think in terms of P&Ls and OKRs. The art is translation: enable real engineering progress and provide signals the business recognizes. Done right, good metrics confirm what strong teams already know – and make trust easier.

Where this article stands in the debate: we reject surveillance and individual output metrics; we combine quantitative signals with qualitative context; and we use metrics to inform choices, not to control people.

IX. How Business Leaders Can Use This

Different strategies call for different lenses. Use this to guide conversations with your CTO.

Safeguards against metrics theatre: publish definitions; show trends instead of points; always pair a hard metric with a soft/qualitative check; declare owners; forbid individual output targets; preview metrics with teams before exec review.

Below, I’m giving a few concrete examples of actions based on the metrics discussed so far.

Accountability legend: [ENG]=Engineering, [PM]=Product Management, [SRE]=Site Reliability/Platform/CloudOps, [CS]=Customer Success/Services, [SALES]=Sales/SalesOps, [MKT]=Marketing/Growth, [FIN]=Finance.

IX.A Revenue Growth

• Watch: Time to market (Lead Time for Changes, DORA), Deployment Frequency (DORA), 30‑day adoption, attach rate (paid options), product‑related NPS delta.

• Ask: Where did we shorten time‑to‑value this quarter? Which releases changed sales outcomes (attach, win rate)?

• If red, then what: [ENG] Improve in‑app onboarding; instrument funnels. [PM] Reduce scope to minimum path to value. [MKT] Release notes/comms to target segments. [SRE] CI capacity checks; caching. [ENG] Trials/entitlements for paid options; surface value moments. [PM][SALES][MKT] Packaging/pricing revision; sales play.

IX.B Cost Control / Margin

• Watch: Support/infra cost per user, CFR, MTTR, cost per adopted feature.

• Ask: Which modules drive run‑cost? Where does quality degrade post‑release?

• If red, then what: [ENG][SRE] Optimize hotspots; caching/CDN; right‑size resources; re‑platform. [ENG] Add contract/e2e tests; strengthen review; progressive delivery. [PM] Clarify acceptance criteria. [ENG] Reuse components; feature flags; reduce scope. [PM] Validate earlier with customers.

IX.C Retention / Expansion

• Watch: NPS delta by product area, share of wallet growth, incidents on top accounts, usage depth of “sticky” features.

• Ask: What changed for top accounts? Which capabilities correlate with expansion?

• If red, then what: [ENG] Hotfix regressions; targeted UX fixes; latency budgets. [CS] Close the loop with detractors. [PM][CS] Co‑design value events with key accounts. [ENG] Deliver enabling capabilities; telemetry for depth of use. [ENG][SRE] Error budgets; incident drills; capacity guardrails.

IX.D NPS Growth & Advocacy

• Watch: Product‑related NPS delta, % positive product mentions in verbatims, Customer SATisfaction (CSAT) metrics.

• Ask: Which product changes moved NPS? Top 3 detractor themes & fixes?

• If red, then what: [ENG] Hotfix regressions; targeted UX/perf. [CS] Close the loop with detractors. [MKT] Improve release notes/change comms.

IX.E Cross‑Sell / Upsell (Expansion ARR)

• Watch: Attach rate (paid options), PQL→SQL conversion, usage depth of upsell features (PQL = Product‑Qualified Lead; SQL = Sales‑Qualified Lead).

• Ask: Which capabilities correlate with expansion by segment? Where does PQL→Sales handoff stall?

• If red, then what: [ENG] In‑product trials/entitlements; metering; nudges. [PM][SALES][MKT] Packaging/pricing revision; sales play. [ENG] Enrich PQL signals and instrument funnels. [CS] Success plans with milestones.

IX.F Time‑to‑Value / Onboarding Acceleration

• Watch: Median days to first productive use; activation conversion (invite → project → first value event); % implementations on time.

• Ask: Where do new customers stall? What dependencies create delays?

• If red, then what: [ENG] Minimum path‑to‑value; prebuilt connectors; templates; in‑app guides; sample data; checklists; success milestones. [CS] Early solution‑engineering; governance cadence. [PM] Clarify value events.

X. For Buyer Decision Makers (DMU): What to Ask

There are 3 key moments where D/C-level buyer personas (B2B customers, system integrators / channel partners, private equity) will want to have quantitative evidence about Engineering Performance: first selection/due diligence, crisis/repeated incidents, and contract renewal.

During my career, I’ve participated to many acquisitions and partnerships: Finance, HR, Legal, Sales / Marketing / Product are all thoroughly scrutinized. Engineering: hardly and often “on paper only”. This results in post-deal surprises and insufficient structuring funds.

On these occasions, depending on the business at stake, the CTO will have to step in.
Yet, it is not always easy to figure out what to ask (client-side) or what to disclose (CTO-side). Here’s a suggested pick:

First selection: look for a Trust & Operations Fact Sheet (last 12 months): Service Level Objectives (SLO) attainment; incident counts and MTTR percentiles; change failure/rollback rate; deployment cadence; security posture (CVE patching time; pen‑test date; SOC 2/ISO 27001 certifications); resilience targets (RTO/RPO); performance SLOs (p95 latency); support SLA attainment and CSAT; median time‑to‑value; deprecation policy.

Renewal and after issues: look for a Reliability & Improvement Report: SLO/SLA trends; top root‑cause categories; corrective actions with dates; reservations or guardrails for errors / budget burn; support backlog aging; forward risks and mitigations.

Buyer checklist: request:

• 12‑month incident history with MTTR percentiles and 2 RCAs;

• median time‑to‑patch critical CVEs;

• concrete deprecation examples; SLO attainment by region with error‑budget plans.

Translate internal → external: keep team scorecards private; publish service outcomes (SLOs, incidents, cadence, patching).

XI. Conclusion: Metrics as a Language, Not a Weapon

Good indicators don’t “prove” performance – they frame decisions. They clarify tradeoffs and show where reality diverges from plans.

CTOs sit between teams who want to build well and execs who must steer the business. The job isn’t to measure what’s easy, it’s to create visibility that supports delivery, morale, and strategy.

• For teams: use signals they can act on.

• For the business: use indicators that trace to revenue, customer outcomes, and risk.

Start small. Choose a few metrics that matter. Improve the conversations around them.

About This Series — A Practical Guide to Reading Engineering Performance

Software engineering carries a unique tension: it’s a major investment, it shapes customer experience, and it determines how fast a company can learn or react. Yet for most CEOs, COOs, CFOs, and PE investors, engineering remains oddly opaque. Activity is visible, but impact is not. Conversations drift between too-technical explanations and too-high-level summaries.

This series closes that gap.

It brings together field experience and established research (DORA, SPACE, DevOps practices) to offer a clear, business-oriented way to read engineering performance.
It answers practical questions such as:

• Which metrics actually help leaders understand what’s happening?

• How do we avoid metric theatre, gaming, or over-interpretation?

• How do engineering signals map to revenue, margin, NPS, and risk?

• What should teams measure to stay predictable without burning out?

• How should cross-functional accountability work?

• What should buyers or investors ask during due diligence or renewal cycles?

Across three parts, the series builds a structured, audience-aware model:

• Part 1 – The Foundations & Team Layer:
  Why metric conversations break down, how to create a shared language, and the team-level signals that drive sustainable, predictable delivery.

• Part 2 (this part) – The Executive Layer:
  How engineering connects to revenue, margin, customer outcomes, and strategic priorities — and how leaders should interpret this information.

• Part 3 – The Collaboration & Buyer Layer:
  How engineering, product, finance, CS, and sales align around metrics; how to read reliability and operational maturity; and how buyers (customers, partners, PE) assess the organization.

This is not a theoretical framework.
It is a practical, tested approach to understanding engineering performance in real companies, under real constraints, with real business outcomes at stake.

If you haven’t read Part 1, here is what you need to know:

• Engineering needs a shared reporting language to avoid confusion with the rest of the business.

• A small set of stable team-level signals helps anchor conversations about flow, quality, focus, and collaboration.

• Metrics at this level are not scorecards — they support decision-making and help teams steer themselves.

• With the right context, these signals strengthen trust across the organization.

Part 2 builds on these principles and moves upward:
How do we translate engineering signals into business signals?

We look at the executive layer: revenue, margin, predictability, cost-to-serve, NPS, and risk.
The goal is to equip CEOs, COOs, CFOs, and boards with indicators that actually reflect how engineering drives the business.

PART 2 – TL;DR

Part 2 explains how engineering performance connects to commercial outcomes — not by inventing new metrics, but by translating technical signals into business language.

Here are the main conclusions:

1. Executive metrics must link to value creation

• The business cares about adoption, expansion, margin, churn risk, and predictability.

• Engineering signals must be reframed in these terms without losing technical nuance.

2. DORA/SPACE still matter – but only when translated

Examples:

• Lead Time connects to time-to-value and faster iteration cycles.

• Change Failure Rate connects to customer trust, credit exposure, and margin protection.

• Deployment Frequency connects to commercial agility and speed of learning.

• MTTR connects to risk in key accounts and reliability narratives in renewals.

3. Attribution must stay honest

• Revenue and NPS are influenced by many factors (pricing, sales plays, seasonality).

• Instead of claiming causality, use contribution ranges, before/after comparisons, and paired qualitative signals (NPS verbatims, ticket categories).

4. Composite executive views are powerful when transparent

• Show the underlying components before the roll-up.

• Publish formulas.

• Use red-rule exceptions (a high composite score cannot mask a failing component).

5. Metrics must reflect strategy

A growing SaaS company, a margin-focused PE-backed platform, and a mature enterprise product unit will not select the same metrics.
Examples:

• Growth: Lead Time, adoption, time-to-value.

• Margin: cost per adopted feature, infra cost per user, CFR, MTTR.

• Retention: NPS delta, usage depth, incident concentration on top accounts.

6. Cross-functional inputs are mandatory

Engineering cannot compute executive KPIs alone – alignment is required.

• Product → usage and activation

• Finance → revenue and margin, finops

• SalesOps → deal tags and attach rate

• Customer Success → NPS, detractor themes

7. A practical roll-out path exists

• Pilot in one product area.

• Two sprints of shadow reporting.

• Train leads on interpretation.

• Keep formulas stable.

• Scale only once adoption is real.

The essence of Part 2:
Engineering only becomes “visible” to the business when technical signals are translated into revenue, margin, and customer outcomes. This translation is not theoretical — it requires cross-functional inputs, narrative discipline, and careful attribution.

IV. The Executive Layer: Impact, Alignment, Risk

At the executive level, indicators should connect engineering to revenue, margin, customer experience, and risk. DORA/SPACE signals are useful when translated into business language and paired with product/finance data.

DORA is the largest and longest running research program of its kind that seeks to understand the capabilities that drive software delivery and operations performance. It was first published in 2013 and has become an industry reference for measuring certain aspects of software engineering.

IV.A Core Metrics with Industry Backing

Benchmarks vs. internal learning. Cross‑company ranges help, but strategy matters (growth vs. margin vs. retention). Treat ranges as orientation; rely on your own trend lines and YoY movement for judgment.

Adoption nuance. For paid options, adoption can map directly to revenue (track attach rate and expansion ARR). For core features, adoption is a leading indicator – pair with NPS verbatims and retention.

IV.A.1 When DORA/SPACE Don’t Fit – Practical Adaptations

A framework, however good, takes some adjustment to the context:

• Low‑frequency or gated releases (regulated/on‑prem): Track release batch size and time‑to‑feature‑flagged beta instead of raw Deployment Frequency.

• Hardware + software or app‑store mediated deploys: Track time from code complete to customer‑available (incl. certification/approval) and rollback capability (don’t include what you can’t control).

• Heavy CAB/change‑approval processes: Add change lead time (CAB‑ready) and non‑prod incident rate to catch issues earlier (or simplify the process 😊).

• Agency/outsourced delivery: Track definition‑of‑ready quality and handoff defect rate across org boundaries.

• Data/ML products: Prefer time‑to‑retrain and drift incident rate alongside DORA.

These adaptations preserve the spirit of fast, safe flow when classic DORA signals are constrained.

IV.B Mini Syllabus – Map Tech Signals to Business Language

Abbreviations (this section): CFR = Change Failure Rate; MTTR = Mean Time to Restore.

Outcome chains (examples):

Phrasing examples by audience:

To a Dev Lead: “PR review time is 0.8d; let’s keep it under 1d.”

To the CFO: “Time to market for a change is now 2 days; fewer rollbacks (12%) lowered credits by €38k last quarter.”

To the Board: “Release quality improved; high‑severity incidents down 30% and recovery is faster.”

IV.C Attribution & Evidence (Causal Hygiene)

• Acknowledge confounders (external influential factors on either cause or effect). Revenue and NPS move for many reasons (pricing, seasonality, sales plays). Avoid claiming causality from a single engineering metric.

• Use simple designs: before/after with a matched control segment, or difference‑in‑differences1 by cohort/region; hold other factors constant when possible.

• Triangulate: pair a hard signal (e.g., adoption) with a soft one (NPS verbatim themes) and a financial view (credits/tickets). When all three move, confidence rises.

• State uncertainty: report contribution, not absolute attribution (“we estimate 20–30% of the uplift aligns with… based on X and Y”).

• Time windows: pick windows that match the product’s adoption curve (e.g., 14/30/60 days) and be consistent.

IV.D Composite Transparency & Drilldowns

• Publish formulas: list inputs and weights for every composite (e.g., Delivery Confidence).

• Show components first: in reviews, display component trends before the roll‑up.

• Red‑rule exceptions: if any component breaches a red threshold, treat the composite as red even if the score is high.

• Drilldown on demand: allow teams to click through to raw metrics.

Section §VI offers metrics and dashboard examples for various situations.

IV.E Some definitions

• Adoption 30d (%) = % of target users adopting a new feature within 30 days.

• Cost per Adopted Feature = Eng. cost / # adopted features that month.

• Strategic Account Feature Coverage = % of ARR-weighted top accounts covered by recently shipped features.

• Infra Cost per User = (Infra + Support Costs) / active users.

• Time to Value = Median days from contract/start → first value event.

• Support Tickets = Monthly count (all severities).

• p95 Latency (ms) = 95^th percentile of request latency (main business endpoints, from telemetry).

V. Predictability, Productivity – and the AI Illusion

Goodhart’s Law reminder: when a metric becomes a target, it can be gamed. Use composites or paired checks (e.g., Lead Time with Change Failure Rate), keep narrative context, and avoid individual output metrics.

Two common executive questions: predictability (fewer surprises) and productivity (value for cost). Recently, a third: “Shouldn’t AI make this dramatically cheaper?”

V.A Predictability – What Helps Confidence

Indicators:

• Task Completion Rate (80–100%)

• Delivery Confidence Score (composite of Lead Time for Changes + PR Review Cycle Time + Task Completion Rate)

• Pull Request (PR – request to review new code before merging towards production) Review Time

• Rework/Return Rate

Questions CTOs should prepare for: What changed between commitment and delivery? What level of rework stems from scope churn, or technical debt?

V.B Productivity – What Engineering Can (and Can’t) Prove

There’s currently a healthy, strong, and ongoing debate about productivity in software engineering, including on LinkedIn. It is an important concern of business executives, relating to margins and the fact that software business is heavy on labor costs until the business has really scaled (i.e. revenues increase much faster than engineering costs).
This warrants a short foreword.

Productivity is the quantity of “output” (usually: revenue – note that # of features is not a good measure of output: if the business can’t sell them, the effort tis wasted) by unit of “input” (for example: engineering costs). While it’s an important measure, the underlying drivers may be complex and interrelated:

• Adding engineers, or upskilling teams, does not necessarily result in a revenue increase via “more” new features – one reason being the increased complexity of team interactions and dependency management.

• Engineering may be (very) skilled and efficient, while their strategic contribution to the business is insufficient or does not materialize. This may be due to lack of customer feedback / flawed product roadmaps, poor / outdated UX, technical debt, or inadequate go to market.

Engineering can measure:

• Cost per Delivered/Adopted Feature;

• Infra/Support Cost per User;

• % time on strategic initiatives.

Engineering cannot prove alone: Feature “rightness” (Product); revenue conversion (Sales/CS/Finance).

Questions CTOs should prepare for: What’s the cost to deliver a feature customers actually use? What % of effort goes to growth vs. technical overhead?

V.C The Myth of AI Productivity Miracle

Bombastic claims about workforce reduction rarely come with data. Controlled studies (e.g., GitHub Copilot RCT, 2023; Stanford HAI AI Index, 2025) show strong gains on simple/greenfield tasks (senior devs), but little net gain on large/complex codebases where debugging and prompting overhead offset wins. Treat AI as a new ingredient – not a shortcut to meaningful delivery. Measure where it helps; use developer satisfaction as a guardrail.

Besides, there are ways to introduce AI in engineering in a controlled and measurable way. Resources aplenty, beyond the scope of this article; search or ask about “MLOps best practice”.

Bottom line: expect accelerated scaffolding, not accelerated delivery (yet…)

VI. Focus and Context: Select Metrics by Strategy

You don’t need every metric. Choose based on context and strategy.
The syllabus in IV.B defines some of these acronyms and metrics.

VI.A Common Patterns to Guide Selection

Core‑Six by Strategy (prioritize before adding more):

• Growth: Lead Time for Changes; 30‑day adoption; Time‑to‑Value; (plus) Attach rate (paid options); CFR; PR review time.

• Margin: Infra cost/user; CFR; Mean Time To Restore (MTTR); (plus) Cost per adopted feature; Unplanned work ratio; Release batch size.

• Retention: Product‑related NPS delta; incidents on top accounts; usage depth of sticky features; (plus) Share of wallet; MTTR variance; SLA attainment.

VI.B Example metrics & targets for a growth Scenario

Executive (Monthly or QBR – Quarterly Business Review):

• Time to market (Lead Time for Changes). <2 days;

• Release quality (CFR) <15%;

• 30‑day adoption >60%;

• Cost per Adopted Feature $8.2k (down QoQ);

• Strategic Account Feature Coverage >70% (ARR‑weighted);

• Developer Satisfaction 4.2/5.

Teams (per sprint/month):

• Delivery Confidence (scope churn) (yellow) 60%-80%;

• Sustainable Velocity. green >80%;

• Ownership Maturity green;

• Unplanned Work 27% green;

• PR Review Time 0.8 days green.

VI.C Example Exec Dashboards (publish monthly)

Let’s have a look at a few mini-cases [MP7] to make it concrete. They’re fictitious, but realistic.
(definitions of the metrics presented can be found in either §III (Part 1) or §IV)

Mini‑case A (growth): After PR review time dropped 2.3d→0.9d and onboarding guides shipped, 30‑day adoption rose 38%→61%; Time‑to‑Value fell 18→10 days; expansion ARR in Q2 was +€70k vs Q1.

(note: 2-weeks sprints – 26 sprints = Jan-Dec for simplicity)

Mini‑case B (margin): Refactoring a hot path cut p95 latency 420→210ms; CFR stayed at 11%; support tickets −41%; infra cost/user −37% over two quarters.

Mini‑case C (incident): Cloud region brownout → multi-region failover. Primary evidence: MTTR ↑, CFR ↑, Support tickets ↑. Secondary: NPS delta ↓

Mini‑case D (incident): Third-party API breaking change. Primary evidence: CFR ↑, Time to market ↑ (while patching), Support tickets ↑. Secondary: NPS delta ↓

VI.D Starter Packs by Maturity (Keep It Simple)

• Starter (first 90 days): Time to market (Lead Time for Changes); CFR; PR Review Time; Unplanned Work Ratio; one business outcome (30‑day adoption or Time‑to‑Value).

• Intermediate: add MTTR; Cost per Adopted Feature (if DRL‑2); Satisfaction.

• Advanced: add Infra Cost/User (DRL‑3), Strategic Capacity %, Share‑of‑Wallet growth (with Finance/CS alignment).

What’s next?

Part 2 explained how engineering contributes to business outcomes and how leaders can build executive-level scorecards.

Part 3 closes the loop by addressing the organisation itself:

• How business and engineering collaborate

• How leaders align incentives across functions

• What buyers (customers, partners, PE firms) should ask

• How to evaluate reliability, trust, and operational maturity in concrete ways

If Part 2 provided the “what” and “why,” Part 3 provides the “how to use this in real situations.”

About This Series — A Practical Guide to Reading Engineering Performance

Software engineering carries a unique tension: it’s a major investment, it shapes customer experience, and it determines how fast a company can learn or react. Yet for most CEOs, COOs, CFOs, and PE investors, engineering remains oddly opaque. Activity is visible, but impact is not. Conversations drift between too-technical explanations and too-high-level summaries.

This series closes that gap.

It brings together field experience and established research (DORA, SPACE, DevOps practices) to offer a clear, business-oriented way to read engineering performance.
It answers practical questions such as:

• Which metrics actually help leaders understand what’s happening?

• How do we avoid metric theatre, gaming, or over-interpretation?

• How do engineering signals map to revenue, margin, NPS, and risk?

• What should teams measure to stay predictable without burning out?

• How should cross-functional accountability work?

• What should buyers or investors ask during due diligence or renewal cycles?

Across three parts, the series builds a structured, audience-aware model:

• Part 1 (this part) – The Foundations & Team Layer:
  Why metric conversations break down, how to create a shared language, and the team-level signals that drive sustainable, predictable delivery.

• Part 2 – The Executive Layer:
  How engineering connects to revenue, margin, customer outcomes, and strategic priorities — and how leaders should interpret this information.

• Part 3 – The Collaboration & Buyer Layer:
  How engineering, product, finance, CS, and sales align around metrics; how to read reliability and operational maturity; and how buyers (customers, partners, PE) assess the organization.

This is not a theoretical framework.
It is a practical, tested approach to understanding engineering performance in real companies, under real constraints, with real business outcomes at stake.

---

PART 1 – TL;DR

Engineering becomes easier to understand when we anchor conversations in a small number of stable, shared signals.
Part 1 sets the foundations: why metric conversations break down, how to avoid misinterpretation, and which team-level indicators help create predictable delivery without harming flow or morale.

Here are the core ideas leaders should retain:

1. Why the conversation about engineering performance is messy

• Engineering often reports activity, while business leaders need movement of the business (predictability, stability, capacity, risk).

• This gap produces avoidable friction: execs feel under-informed, tech teams feel pushed, and everyone loses time translating.

2. Metrics work only when they act as a shared language

• Signals should not function as proof or scorecards.

• They help people make choices and understand trade-offs.

• Clarity requires ownership, stable definitions, and a common narrative.

3. A disciplined approach prevents gaming and confusion

• Use paired checks (e.g., Lead Time with Change Failure Rate) to keep behavior healthy.

• Keep the set small: five to seven signals, and up to three targets if needed.

• Focus on trends, not snapshots.

• Explain changes before presenting them to execs — trust grows upstream.

4. Build trust around data quality

• Each metric must have an accountable owner and a producing team.

• Acknowledge when accuracy is approximate (manual exports, incomplete tags).

• Show the path toward improved reliability.

5. Team-layer metrics shape execution without burnout

• DORA (DevOps Research & Assessment) covers flow and stability (Lead Time, Deployment Frequency, Change Failure Rate).

• SPACE (Satisfaction, Performance, Activity, Collaboration, Efficiency) adds collaboration, focus, satisfaction, cognitive load.

• Team-facing signals help with planning and self-steering — not surveillance.

6. Composite signals help when ingredients are solid

• Delivery Confidence, Sustainable Velocity, and Ownership Maturity summarize complex realities without oversimplifying them.

• They help teams and execs read progress consistently.

7. Metrics can harm if misused

• Coverage quotas trigger shallow tests.

• Velocity targets break predictability.

• Zero-defect expectations create big-batch behavior.

• The fix: pair metrics, reduce pressure, and focus on decision support.

8. Indicative benchmarks help, but context wins

• Benchmarks offer orientation, not judgement.

• Internal trend lines matter more than external comparisons.

The essence of Part 1:
Engineering becomes more transparent when the organization agrees on a small set of team-level signals and interprets them with context, narrative, and shared accountability. This is the groundwork for all executive-level reporting in Parts 2 and 3.

Some background.

The subject (Software Delivery Performance) has been discussed at length for 60 years: software engineering is costly and complex, it’s not unreasonable to seek some assurances about delivery dates, quality, costs, or user adoption, or ”insert business or social concern here”.

Project management in, and simply management of software engineering were actively developed since then, starting with proven approaches typical to construction or industry, continuously feeding from other domains (e.g. embracing Lean) and pioneering original methods (e.g. Agile flavors) later retrofitted in other domains.

IT can be usefully compared with Construction or Manufacturing: it has platforms just like cars. It has pipelines like assembly lines. It has architecture. It has a supply chain and components. It has similar concerns about productivity, speed to market, or ROI, while having quite different constraints (e.g. little Cost of Capital vs. high Personnel Costs).

Consequently, there has been continuous efforts, experiment and research so as to best assess productivity & predictability of software engineering, and eventually methods to secure and improve them, is thus legit and as old as the IT industry itself.

And, the debate has been occasionally heated, due to: business pressure on costs, customer pressure on quality, experts’ “clerical” debates, marketing overpromises – to name a few.

Why this matters?

Every year, we get studies from the likes of Gartner, Forrester, IDC or McKinsey (to name a few) which report that, depending on the project type (engineering vs. integration) and size, planning and costs are off by a vast margin (30%-60%), with scope issues in the middle.

Recent marketing lullabies have been promising to reduce software engineering headcount drastically thanks to AI.

A personal experience I think many other CTOs / Engineering VPs will recognize: in a weekly management team meeting of a software business, the sales leader and the CFO report with known, sharp and simple metrics, and can drill down on the spot if needed.
Conversely, the CTO often sounds very qualitative and occasionally vague, or over-complicated.

In my consulting or interim exec engagements, I hear almost every single CEOs of software businesses voicing repeatedly 3 concerns about their engineering team (even if there’s no crisis or difficulty):

• Does the business get enough for what it costs?

• It is a black box (understand: “as opposed to sales”), with various underlying concerns: predictability, continuous improvement, risk management.

• How can it be more predictable (more transparency about timing, scope, quality)?

This is avoidable. The aim here isn’t to coin a new theory – it’s to collect what works, define it clearly, and show how to use it so CTOs, CPOs, their teams, and their peers can hold better conversations and make better calls.

Let’s see how we can equip ourselves with ways to find answers.

Disclaimers

• Nothing here suggests people should be managed by numbers. Good leadership first, metrics second. Actually: many talented and experienced technical leaders can “smell” where the problems are, while metrics help making assessments objective or, at least, less opinionated.

• No silver bullets: Metrics are signals, not proofs. Attribution from engineering changes to revenue/NPS is not straightforward – there are various other influencing factors / confounders; I offer outcome chains and evidence patterns, not lab‑grade causality.

I. A starting point.

Ask any CTO how productive their engineering team is, and you’ll often hear: “We’re shipping regularly, our core metrics are stable, (+ details of the moment.)
This doesn’t tell a CFO or CEO how engineering is moving the business. The gap between technical progress and business impact creates friction: engineers feel scrutinized; executives feel under‑informed; CTOs translate between worlds.

On March 11, 2025, Matthias Patzak published “A CTO’s Guide to Measuring Software Development Productivity” on the AWS Enterprise Strategy Blog. It’s a solid foundation (multi‑dimensional metrics; outcomes over activity; avoid individual scorecards). This series takes the next step: turning that foundation into a practical, audience‑aware model and a set of scorecards that connect engineering to business outcomes.

The post groups metrics into five areas:

1. Delivery performance (e.g., certain devops metrics)

2. Code quality (e.g., defects, rework)

3. Team health (e.g., engagement surveys)

4. Operational excellence (e.g., incidents, Mean Time To Restore)

5. Business impact (e.g., adoption, usage)

It also recommends: treat metrics as signals (not scorecards), focus on trends (not snapshots), and avoid individual‑level productivity measures.

This is a good baseline, but:

• There are no concrete metric sets proposed (was not the article’s intention), it’s more about directions to take or avoid.

• Some objections can be made, for example:

  • code quality signals need context, are about output and can be gamed.

  • There’s an overlap between Delivery vs. Operational metrics depending on who owns them in the organization.

  • Surveys are just one way of understanding team health. Think of unplanned work ratio, decision latency, and participation in reviews/retros.

• Business impact needs definition. Turn “impact” into adoption, margin, NPS trends, and post‑release revenue signals.

In the following section, I’m building a model for selecting who sees what, how metrics are computed, and how technical change maps to business contribution.

II. Metrics Are Shared Accountability

Adapting metrics to audiences isn’t novel – but friction persists. Common patterns I’ve observed or lived – to list a few:

• Executives browsing Jira/analytics and jumping to conclusions without context.

• CTOs reporting “green” output metrics without explaining business impact.

• Business leaders debating technical indicators (e.g., test coverage) without engineering background.

• Developers pushing back on KPIs they can’t influence.

To avoid these pitfalls:

• Treat metrics as a shared language (context, purpose, narrative)

• Set guardrails

• Secure data quality and ownership

By ”shared language” I mean:

• Share indicators a group can act on, or own, so that they can take pride in success.

• Provide context and narrative, not just numbers. Engineering metrics, like financial indicators, require a narrative.

Guardrails to keep metrics healthy:

• Avoid individual output targets, measure at team or product level and favor outcome over output.

• Counter gaming or provide nuances with composites or paired checks (e.g., Time to market with Change Failure Rate), plus narrative context.

• Keep 5–7 signals and ≤3 targets; if more are needed, place extras in an appendix and review their impact on / contribution to decisions.

• Preview metric changes with teams before executive reviews.

• Keep composite formulas stable for at least two quarters so people learn how to read them.

• Report only metrics that matter in the situation (i.e., those which help deciding how to achieve more impact and better outcomes in your current situation). You can maintain a broader set regularly calculated, no problem, just avoid confusing audiences with an excess of information that’s actually not useful.

• Change a metric only if it’s flawed or no longer tied to company goals – not to rotate for its own sake.

Data readiness & accountability: never propose what you/others cannot measure reliably. Each metric should have an Accountable owner and a Producing unit (e.g., Engineering produces lead‑time; Finance produces margin).
If you miss a metric from other departments to calculate your own ones, don’t try to palliate – that won’t be trusted.
§VII.D in Part 3 offers a simple Data Readiness Levels (DRL) ladder to help you manage metrics production.

Conclusion: sharing metrics is not just reporting: it is culture and ethics.

III. The Team Layer: Execution Without Burnout

Team‑facing metrics are for self‑steering and learning – not surveillance. They improve predictability, reduce friction, and protect sustainability.

In the following sections, you’ll often see references to “DORA” (DevOps Research and Assessment). In case you don’t know: it is the largest and longest running research program of its kind that seeks to understand the capabilities that drive software delivery and operations performance. It was first published in 2013 and has become an industry reference for measuring certain aspects of software engineering.

III.A Recommended Team Metrics

Combine DORA with SPACE (developer experience: Satisfaction and wellbeing, Performance, Activity, Communication and Collaboration, and Efficiency and flow). The list below offers a subset, together with indication of a desirable target, and the consequent positive outcome:

• Time to market (Lead Time for Changes, DORA): <1 day (elite) ⇒ fast iteration.
  Def: time from first code commit to code running in production.

• Release quality (Change Failure Rate, CFR, DORA): <15% ⇒ stable quality.
  Def: % of deployments causing incidents/rollbacks.

• Deployment Frequency (DORA): daily/weekly ⇒ CI/CD maturity.
  Def: # of deployments in product per day / week.

• Pull Request (PR) Review Cycle Time (SPACE): <1 day ⇒ responsive collaboration.
  Def: time from 1 opened (or equivalent if you have a no-PR process) to merge/close/pass (calendar time)..

• Task Completion Rate (SPACE): 80–100% ⇒ planned work delivered.
  Def: Completed / committed in the sprint.

• Satisfaction & Focus (SPACE): >4/5 on internal surveys.

• Cognitive Load (SPACE): <3/5 ⇒ manageable scope.
  Def: self‑reported 1–5 scale of mental load.

• Unplanned Work Ratio: <25% ⇒ sustainable focus.
  Def: unplanned effort / total sprint effort (by story points or hours as per tickets status history).

• Task Bounce Rate: <10% ⇒ clear ownership.
  Def: % of items that change owner/status due to unclear scope or dependency.

III.B Composite Signals That Work

As previously explained, it is not recommended to analyze metrics in isolation. An option is to combine some of them into a composite signal, in the spirit of checks-and-balances.
Here are some examples you might find useful for your situation:

Compute composites only from well‑defined inputs, normalize to a 0–100 scale, weight them (or not) according to your situation, and keep the formula stable for at least two quarters so teams learn how to read it.

A Story from the Field (Fictional)

At SteelWallet, a fintech platform, leads saw long lead times and inconsistent PR reviews. Rather than pushing harder, they instrumented a subset of DORA + SPACE across five teams and added async reviews with SLAs. They reduced unplanned support work by tagging and triaging it at stand‑up. Within a quarter, Time to market fell 40%, PR review time halved, and satisfaction rose from 3.4 to 4.2. The numbers didn’t replace judgment – they improved retros and trade‑off decisions.

III.C Other Considerations

Measuring Subjective Signals Well (e.g. Cognitive Load, Satisfaction)

• Use anchored scales (e.g., 1 = overwhelmed, 3 = manageable, 5 = easy).

• Keep question text stable for trend validity; rotate only a small bank each quarter (if you have to).

• Report on median and interquartile range (IQR), not only averages.

• Aim for ≥70% participation; record “prefer not to answer” to avoid biasing results.

• Pair with a qualitative prompt (“What made this sprint harder/easier?”) and summarize themes in retros.

When Metrics Make Things Worse, and How to Fix It

• Set a coverage target: you get superficial tests.
  Fix: drop raw coverage as a target; track post‑release defects and review health; consider a composite that includes mutation score2 or critical path test time.

• Set Velocity quotas: you get scope slicing & rework.
  Fix: avoid team output quotas; use Delivery Confidence + Rework/Return Rate + Unplanned Work Ratio as a check‑and‑balance triplet.

• Push for “Zero Change Failure Rate”: you get deploy fear & risky big batches.
  Fix: pair CFR with Deployment Frequency and Release Batch Size; adopt progressive delivery and error budgets (capacity reservations in case things go south).

• Ask to Close Pull Requests in 24h: you get rubber‑stamp reviews.
  Fix: pair PR Review Time with PR Comment Depth (or review participation) and post‑merge defect rate.

Principle: prefer composites or paired checks. That’s general advice by the way: for example, investors always consider Internal Rate of Return and Net Present Value together to assess the quality of an investment.

And of course: Guardrails from section II do apply.

Benchmarks (Indicative) from the industry

While context (type of software, type of delivery, etc.) varies, a typical SaaS team would strive for:

Governance for Team‑Defined Metrics (Lightweight)

Of course, Teams may define additional metrics for their respective needs and peculiars, and I advise to do it under a basic governance:

• Not for individual objectives or ranking.

• Provide a clear rationale and definition.

• No “official” sharing beyond the team.

• If promoted to a team objective, an n+2 review validates intent and safe use.

• Respect the 5–7 signals and ≤3 targets guidance – avoid drowning debates in detail.

Data Readiness Levels (DRL) & Accountability (Quick Check)

Some of which are already mentioned in §II (Guardrails):

• Don’t propose what you can’t measure reliably.

• Each metric has an Accountable owner and a Producing unit (e.g., Engineering produces lead time; Finance produces margin).

• If accuracy is DRL‑1 (manual ±20%), label it and avoid punitive use; plan an upgrade path.

• See §VII.D in part 3 for the DRL ladder and examples.

• Small and Stable: Keep a small, stable set (5–7 signals; ≤3 targets). Retire only when flawed or no longer tied to strategy.

• No surveillance: Avoid individual output metrics. Measure at team/product level and pair numbers with narrative context.

• Only propose what you can measure: each metric needs an Accountable owner and a Producing unit.

What’s next?

Part 1 focused on the foundations and on the team layer — the level where flow, stability, and sustainable execution are shaped.
Part 2 takes the next step: how engineering performance maps to business outcomes.

We move from:

• “Are teams executing predictably?”
  to

• “How does this execution affect revenue, margin, and customer experience?”

If you want to understand the bridge between engineering operations and business performance, Part 2 is where that connection becomes practical.

1. What this publication is — and how to read it

The Software Factory Brief is not meant to be read from top to bottom.

Rather, it’s a growing collection of long-form articles about how software organizations operate once scale, pressure, or complexity start to shape day-to-day decisions — and why the same issues tend to reappear across different companies.

A few points of orientation before you start reading:

• articles are deliberately long and dense

• most pieces are written to be read *when a situation arises*, not when they are published

So jump directly to what matches your current context, that’s how it’s meant to be used. Bookmark some articles for your later reference.

2. How the content is organized

2.1 Core themes (tags)

Articles are grouped around a small number of recurring themes. These are not short-lived topics, but structural questions that tend to surface repeatedly in software companies.

You’ll find pieces organized around themes such as:

• Product–Engineering Alignment
  Decision rights, roadmaps, incentives, and what actually drives alignment — or misalignment — between product and engineering.

• Scaling Software Organizations

  What starts to break as teams grow, and why adding people or process rarely addresses the core problems. Near- and Off-shoring, follow-the-sun, multi-cultural organizations.

• Software Quality & Technical Debt

  How quality degrades over time, how debt accumulates quietly, and why it eventually becomes a leadership concern.

• Portfolio & Product Strategy

  Flagships, overlap, rationalization, and the long-term cost of carrying products that are no longer questioned.

• Leadership & Governance

  Postures, roles, coaching, organizational interfaces, metrics, and the operating model behind executive decisions.

• Operations, Support & Delivery

  Support load, professional services, reliability, and the economics of running software at scale.

These tags are meant to be used as chapters, not as labels. If one theme is relevant to you, it’s usually worth exploring the related pieces in the same area.

2.2 What “long-form” means here

Most articles are closer to working notes than to opinion pieces.

A typical piece will:

• start from a concrete situation

• examine the underlying mechanics

• make trade-offs explicit

• provide concrete, exemplified approaches to deal with the issue, no one-size-fits-all

They are written to be:

• bookmarked

• reused in internal discussions

• revisited when a similar situation occurs or comes back

If you’re looking for short reads between meetings, this may not be the right moment.

If you’re dealing with a real issue and want clearer thinking around it, this is the intended use.

3. Where to start, depending on your situation

Many readers arrive with a specific question or source of tension in mind, so I’ve listed a few starting points, depending on what you’re dealing with.

If you’re facing scaling or growth pressure

Typical signals:

• engineering output slowing down despite increasing headcount

  (often not measured appropriately)

• delivery becoming less predictable

• more processes without more clarity, people complain about bureaucracy

You may want to start with:

• Software engineering: what could go wrong? (a 3-part series, here’s the direct link to part 1)

• Why software engineering doesn’t scale the way leaders expect (planned)

• Software Delivery Performance (SDP): A CTO’s Guide to Building Trust and Clarity Through Metrics (a 3-part series, being published now)

These pieces look at the structural failure modes that tend to appear as organizations grow — often before they show up clearly in metrics.

If product and engineering are drifting apart

Typical signals:

• roadmaps that look coherent but stall in execution

• recurring friction between business, product, and technical leadership

• priorities changing frequently, without explicit trade-offs

Good entry points are:

• The CEO–CTO divide: where misalignment really starts

• Why backlogs become graveyards — and how to bring them back to life

These articles focus on decision rights, incentives, and the mechanics behind alignment — not on roles, rituals, or org charts.

If quality, reliability, or incidents are becoming an issue

Typical signals include:

• defect levels increasing release after release

• support load growing faster than usage

• engineering time increasingly spent on firefighting

• not-so-great NPS & customer satisfaction, low adoption

You may want to start with:

• Quality crises don’t happen overnight — they grow in the shadows (planned)

• Why technical debt eventually becomes a leadership problem (planned)

The emphasis here is on understanding how quality erodes systemically, and what leaders often miss until the cost becomes visible.

If you’re dealing with portfolio complexity, restructuring, or M&A

Typical signals:

• overlapping products or features

• “flagship” products that no longer justify their cost

• integration efforts losing momentum after the deal

• product ROI unclear, challenged and possibly hopeless.

Relevant starting points are:

• Beyond the flagship: when product portfolios quietly drift

• Why post-merger integration fails without a product/engineering/ops lens (planned)

These pieces look at portfolio and integration decisions from an operational perspective, not only a financial or organizational one.

If you’re exploring how software organizations work more broadly

If you don’t have an acute issue but want an overview of the themes explored here, you can start with:

• My ideal software company (planned)

• Threshold effects while a software business grows (planned)

From there, follow the themes that are closest to your current context.

How this section will evolve

As the archive grows, this page will be updated to:

• reflect the pieces that are most often referenced

• clarify reading paths

• surface articles readers tend to return to

It’s meant to function as a **living index**, not a fixed reading list.

4. What to expect next

This publication grows slowly, by design. Maybe I’ll make a book out of it one day - could be a fun project.

Over time, I’ll continue to explore questions that tend to surface once software organizations move beyond early growth, such as

• trade-offs between reliability, speed, and cost

• recurring tension between product intent and delivery reality

• the interaction between quality, ops & support load, and engineering productivity

• portfolio and platform decisions that look reasonable locally but fail at scale

There is no fixed roadmap and no attempt to cover everything.
New pieces usually emerge from situations I see repeating across different contexts.

5. How to engage

If you want to engage, the simplest way is to reply directly to an article.

Replies are treated as conversations, not as comments to moderate. Typically, we (you, other subscribers, and I) are interested in:

• clarifications

• disagreements

• “we’re dealing with something similar” notes

• suggestions for future topics

• authoring collaboration suggestions (I have a couple ideas where I need help)

Many articles start from questions or patterns that surfaced in exchanges like these.
Of course, you can also read quietly and use the pieces as internal reference material.

6. Continuity

If you arrived here because a piece was shared with you, subscribing is simply a way to maintain continuity.

You’ll receive:

• one or two articles per month

• by email

• with no expectation to read them as they arrive

Each post is written so it can be useful weeks or months later, when the context matches.

If this fits how you prefer to read and think, subscribing makes sense.
If not, feel free to visit when something is relevant.

Every product team has them – those backlog items older than a year, untouched and slowly sinking into irrelevance. But what if those forgotten ideas held untapped value?

There are plenty of best practice articles on product backlog management. However, I notice they don’t really address one point I find important: what to do with all the ideas (internal, from customers) that accumulate over the years and sit untouched? What they say is: “dump them, if nobody asked then it does not have value, and avoid clutter in the first place.”

I agree with that: a backlog is not an archive or a graveyard of once-good ideas. However, I think all product teams create oldies, and there is value in these items. We can do much better; let’s see how.

The issues

Without structure, your backlog becomes a liability: wasted product owner time, lost revenue from unimplemented value, and declining user engagement.

Setting the scene

Software businesses have product release cycles: companies want/need to make buzz at regular intervals to raise customers’ and prospects’ attention – and that’s fine. This is reinforced by seasonal trade shows and market analysts’ publications. On the side: it’s not antagonistic to Continuous Deployments: feature locks can manage this.

Another source of seasonality is product roadmaps, which plan for the next 2-3 years and the next 12 months in rolling quarters.

Lastly: the fiscal year itself is a natural heartbeat.

Put all this together, and you get 3-months iteration planning, with several sprints within.

Issue 1: User Stories graveyard

Thus, user stories (lower-level backlog items) that do not make it (to sprint planning) within 12 months sink to the bottom, never to resurface. This is wasted time and effort: producing backlog items doesn’t come for free. When product owners and tech leads work on them, that’s less bandwidth to answer clarification questions from developers, testers, or technical writers.

On the other hand: producing new ideas is a key motivator for product owners and tech leads

→ A balance is needed.

Issue 2: Customer feature suggestions / change requests archive.

In this case, I’m discussing the small items, not the strategic product directions you get from your key customers through Customer Advisory Boards (CABs) and such.

Whether or not suggestions or requests are consistently and regularly curated and transformed into user stories, many product and engineering organizations don’t implement most of them, if at all - the product roadmap primes. These are wasted opportunities.

Satisfying faithful customers through quick wins and quality-of-life changes is a great way to keep them engaged, with positive effects on customer satisfaction and NPS.

→ A balance is needed.

Let’s see how such a balance could be achieved.

A plan of approach

This isn’t about cleaning up Jira. It’s about recovering business value, increasing retention, and reducing churn.

I’m proposing that you take a data-driven, quantitative, and qualitative approach to make your “oldies” provide business outcomes. This will take time and budget and will trigger resistance, so prepare to sell the plan and manage change – I’ll come back to it at the end.

Here’s the idea:

In what follows, I’ll be using “feature ideas” to refer to customer change requests, feature ideas, and internally generated user stories older than 12 months. The content to review and value is quite granular.

1. Clean the backlog

Step 1: Reduce your product backlog.

Make your active product backlog “operational” (by active, I mean: which the engineering teams consume on a “sprintly” basis from e.g. Jira). It should have a simple hierarchical structure, with the Epics of the rolling 4 quarters at the top. Unless your product has a broad functional span and your engineering organization is really large (e.g. 500+ FTEs), you probably don’t need the Initiative / Theme level.

Manage your operational backlog like you manage emails: after each iteration, move what was finished in a mirror “Done” backlog. This one can grow forever, it will be mostly used to check for regressions or security certifications.

Step 2: Build a Feature Ideas Archive

Move every feature idea that was not “done” in the past 12 months (or whatever limit you want to set) into an external database (can be an Excel sheet...) which product owners, product managers, and tech leads can slice and dice in a simple way.

Step 3: Augment your Feature Ideas Archive

Augment this database, which I’ll be referring to as the “Feature Ideas Archive” below, with customer change requests or feature ideas that you typically find in your customer support portal and other sources (user forums, internal individual treasure troves from Subject Matter Experts (SMEs), support reps, consultants, etc.). By “augment” I mean: copy. Leave the originals where they belong, and just make sure you maintain synchronization in the future.

Step 4: Curate your Feature Ideas Archive

Finally, curate the database: remove duplicates or similarities (e.g., change requests translated to user stories in the past). Remove those that don’t make sense at all (e.g., so old that the product has taken a different direction since then).

2. Map density

Step 1: Map the feature ideas.

Map the feature ideas archive to the reference functional decomposition of your product.

This is the hierarchical tree of functions and features your product has – tip: it is close to the outline of your user reference documentation.

Side note: I’ve observed that Product / Engineering organizations don’t have such a formal tree, and this is a problem, in my opinion. For one, recent hires don’t have a quick reference and will get lost, make assumptions, or burn experts’ time with questions. For two: this means that you don’t know your functional test coverage at system level, which can be a major problem to assure quality. So, if you don’t have one, get one fast. A simple way is to start with the outline of your reference manuals, decompose it down to 4 to 5 levels, ask your LLM of choice to review and complete it, and ask your SMEs (customer support, professional services) to review.

Initializing this mapping will be a substantial effort, but it’s worth it. Keeping it up to date is a small effort.

Step 2: Analyze the mapping.

Once you have the mapping, you will be able to see where your product is challenged the most. Of course, I’m sure you will only confirm what your intuition already told you! But what’s important is: you will know how much. You will have indisputable quantitative insight into the state of your product.

In practice, you will be looking for 2 key information in the first 3 functional levels:

1. What is the % of ideas vs. existing features? 10% to 20% is healthy, less than 10% is either ok, or a sign that users don’t care about this function or that your product owners don’t have the time to do regular due diligence and creative work. More than 20% either means that your function has big gaps, or that product owners/managers are not busy enough ;)

2. What are the top 5 / top 10 product areas with feature ideas, possibly weighted by age and some business score? These will be areas to focus on to delight your user base.

3. Overlay telemetry.

Another source of quantitative information that can tell you where to add to your product is telemetry. Most of you already take advantage of it, but for the sake of others, let’s discuss it briefly.

Your telemetry information tells you about the users’ journey in the product. Thus, you’re able to segment it across the functional decomposition I was explaining above and harvest insights such as:

• Most frequent product usage patterns and most used features

• % of product usage time spent doing certain tasks

• % of time spent by main usage pattern vs. your normalized UX journey time

• % of drops (users interrupting a task and not resuming it) per use-case or function

• Frequency of errors/warning messages per function

Such indicators will guide you to value your feature ideas archive, especially related to quality-of-life improvements. Consider this:

• Function A is the second most-used product top-level function

• Inside function A, 70% of users’ time is spent performing use-case A.1

• The average time spent performing A.1 is 30% above your reference UX test time for A.1, with a standard deviation of 15%

Discuss this finding with your support organization (Does it correlate with usage issue tickets? Do they recognize the pattern themselves?) and your customer training team (Did they observe trainees struggling with this use case?). Depending on their feedback, it may be a clear sign to explore quality of life improvements from feature ideas mapped to function A. Those improvements, or their hints, have likely already been reported.

Correlating feature ideas density and telemetry insights will give you a valuable, data-driven focus on what truly matters to your users.

4. Engage & sustain

4.1 Engage users

You can get much more insight from your user base.

You will likely find, to your frustration, that feature ideas in your Archive are too vague (precision or rationale). This is a call to gear up your user/usage information gathering. Here are a few ideas - you may already have them running in whole or in part. You don’t have to do it all - start small.

• Create a moderated user forum inside or next to your company’s extranet or within your support portal. Make it accessible directly from your product with a link (help or contextual menu).

• Run a subreddit for your product. Make it accessible directly from your product.

• Add a Slack (or similar) ideas chat directly in the product, next to the “help, I’m struggling” support chat.

• Use LLMs to rewrite user suggestions in a more condensed / crisper way.

• Hold user gatherings at each key external event (trade shows and your own users’ conferences), and have an “idea board” on the side where anyone can suggest feature ideas without interrupting the sessions.

• Add a “Suggest improvement” feature next to the “Report bug” one inside your product, with a simple screenshot & text/drawing feature integrated, along with journey summary and log gathering - asking permission to share with appropriate anonymity.

• Centralize all ideas sources on a single web “page” with convenient browsing and search, and offer users to vote. Gamify it and offer incentives such as extra usage credits, etc. Host online thematic voting sessions where the most active users moderate. Also, having a single ideas repository will help you have a single source to synchronize with your Feature Ideas Archive.

• Make sure your Customer Advisory Board is well structured: next to the usual Exec / VP-level meeting, have a dedicated experts’ group with your SMEs and some field experts from your key customers, and a Technical Advisory Board if your customers are corporate companies with complex IT requirements.

4.2 Keep your promise and sustain

Very important: This is a journey without a return.

Once you’ve started this journey, which will take budget and resources (see below), be conscious that you cannot stop, and you owe users regular feedback and tangible proof that you’re taking their input into account. It’s like authoring on LinkedIn or starting your YouTube channel: if you stop or slack off, you will lose your supporters and subscribers very quickly, and most will not come back if you resume.

In other words: if you don’t sustain, the impact on customer satisfaction and NPS will be important and hard to recover. But the rewards are certainly worth it, as we’ll see.

Users will make the effort to give you feedback if they get something in return. In the beginning, traction will be slow; they will behave as “constructively skeptical”: they will give it a try and give you a chance to prove yourself.

Thus, it’s important to be conscious of what it takes to keep the promise and sustain this initiative over time: transparency, feedback, budget, resources, and commitment.

4.3 Transparency and feedback

Stay true: even if early results are disappointing, demonstrate you are bold and committed.

Next to giving access to your Feature Ideas Archive to all users, I suggest that you publish broadly (i.e. even outside your paying customer base) some basic statistics every month or so, such as: # of new feature ideas captured in month, # of implemented feature ideas in month, average lead-time to implementation, % of feature ideas discarded, etc.

Next to that, just like in regular customer support, set yourself a feedback obligation SLA, such as:

“Any new feature idea is answered within 2 working days with either:

1. Great idea! We will deliver in the coming 3 months or sooner.

2. Good idea, planned within 12 months.

3. We’re quite full already; your idea is #246 (next 12 months’ list currently ends at #185).

4. We have carefully considered your idea and have decided not to pursue it. We understand your disappointment but encourage you to continue providing feedback to us and the community.”

Finally, inform each reporting user when their idea has been implemented in the channel where they originally reported it, and make sure to update its status in the Feature Ideas Archive.

4.4 Resources and commitment

This is about one of my favorite personal aphorisms: “what does it take to be serious about it”™

What I’m describing above doesn’t come for free. Here’s a brief list of what you’ll need to make it successful.

• A community manager dedicated to user forums / subreddits / slack and monthly reporting to the community.

• A fraction of time from your customer-facing product experts: support reps, consultants, SMEs. They will have to review user comms channels daily, on a rotating basis, to either guide users in best achieving their goals (there could be ways to achieve what they ask for, even if not the way they thought about, or not the “best” way) or to capture a feature idea in a clear, non-ambiguous and actionable way. In order to facilitate change internally, I suggest that this is funded from the Product budget back to the contributing departments.

• Around 20% of the time of one of your product owners to curate, maintain, and drill through the Feature Ideas Archive

• An aggregation portal, some BI development, and LLM costs to analyze verbatims.

Obviously, you don’t have to go full-scale from the beginning. What I’ve listed above is more like your “dream state”. Start small and roadmap it; what is most important is to sustain the effort in time. Here are some minimal starting cost figures (EU standards):

• Community manager (someone with a reasonable understanding of your product and field): 75KEUR fully loaded.

• Initial tooling: 20KEUR

• Internal costs (left pocket - right pocket, it is not cash-out, but will become later on as you will consume existing resources up to a point that additional hires will be necessary): ca. 20KEUR

Total initial investment: ~120KEUR. Keep this in mind until I discuss the ROI.

Lastly, I advise that you establish clear guardrails (i.e., engineering bandwidth) in each sprint to make sure that these initiatives are not sidelined. In clear words: in each sprint, each team should deliver at least one vetted feature idea. This means integrating feedback processing (e.g., from customer support) into iteration planning. By making it a non-negotiable part of the process, you ensure that user insights are continuously acted upon, maintaining momentum and delivering ongoing value.

5. Measure impact

Finally, here’s my value equation! We need to find proof in the pudding.

To measure the impact of your Feature Ideas Archive initiative, track changes in Customer Satisfaction, Customer Retention, and NPS that can be directly attributable to the regular, small product improvements (up to changing your quarterly and spot satisfaction surveys as needed).

I could not find research specifically correlating CSAT and NPS increase to revenue increase in B2B SaaS, but various correlation effects have been largely documented since the mid-2000s: NPS increase demonstrably leads to upsell increase, customer churn decrease, CLV increase, and CAC reduction.

Still, it’s not unreasonable to do some back-of-the-envelope calculations. Let’s suppose:

• Working on the “Feature Ideas Archive” results in a 5% increase in product-related customer satisfaction within 6 months.

• A 5% increase in customer satisfaction translates into a 2% increase in retention or upsell opportunities – sounds reasonable.

• For a company with 1,000 customers and an average annual revenue of $15K per deal, this could mean $300K in additional ARR.

That’s an ROI within the fiscal year. In other words: it’s worth a try

If you still struggle to sell the plan, you can refer to some well-known examples in the software industry. Companies like Slack, Adobe, and Atlassian are quite good at nurturing customer feedback, and there’s abundant literature about their approaches. Slack’s user forums and beta programs create a direct line to their community, enabling them to quickly iterate based on feedback. Adobe’s customer advisory boards and forums help prioritize features that matter most. Atlassian, through its community and transparent roadmap, ensures that users feel heard and valued.

Conclusion

Dormant feature ideas, whether from internal or external sources, often clutter product backlogs. If you keep archiving them where they are “just in case”, you’re wasting effort and revenue opportunities.

For many teams, this can be a sub-150KEUR initiative that returns its cost in-year, while re-engaging both customers and internal teams.

What’s key is: once you start, it’s a journey without return: you and your organization need to commit.

I hope I was able to come across and some of you readers will start advocating such an initiative in your company. If you’re considering it, let’s connect. I’m keen to hear how others are solving the same problem – and happy to share lessons learned in more detail.

I hope you have a rewarding journey!

In my previous articles in this series (Part 1: People and Part 2: Process and Technology), I’ve compiled a list of blockers, pain points, and issues that you typically meet in the engineering department of a software company or business. In this third part, I’m outlining how you can structure the repair journey, trying to provide as many pragmatic tips as possible while keeping this article digestible. You can use this advice w/o reading the first two parts, but I’d recommend it still so that you understand where I’m coming from.

Before I start, let me clarify the setup considered here: you’re the CTO of a software company or business unit with a 50+ strong engineering team, up to a couple of hundred. You’ve just taken your position, been promoted or externally hired, or made the firm decision to change the game.

Preamble

To assess the situation and build the adequate transformation plan, you will want to gauge 3 key dimensions:

• People: what’s the engineer’s morale/team spirit?

• Efficiency: what’s the delivery flow from User Stories to customers experiencing them? (I’m focusing on engineering here; the journey from customer needs to User Stories is another potential can of worms)

• What’s the product quality as experienced by end-users (at customers, partners and internally: Customer Support, for example)

• What’s the level of technical debt?

Let’s plot team morale against quality and efficiency:

You need to know where your engineering department is in this chart. This will help create a sense of urgency, the need for change, and hint at where to start with. Roughly speaking: your goal is to improve customer satisfaction. Hence, product quality is your target. For that, you need to improve efficiency. This will start improving team morale. This virtuous circle will progressively allow resuming innovation with increased speed to market, which will also positively impact customer satisfaction and team morale.

What’s fundamental is that Team Spirit is influenced by every other element and influences them back.

The proposed steps below proceed chronologically. Some steps can or will have to be undertaken in parallel. Don’t try to do everything at once though, especially when it comes to how things change – rushing it will trigger resistance and slow the whole journey.

Let’s assume the worst: dispirited teams, a quality crisis, and piles of technical debt both in tools and product. If parts of this don’t apply, skip them.

Stage 1: Diagnostic and master plan (first quarter)

In this stage, you will want to understand how acute issues are and which one to prioritize.

Step 1.1

In an engineering all-hands meeting, announce that you will engage in the improvement journey, reminding all that the situation is not sustainable for the company. Share the next 4 steps (1.2 to 1.5) below and:

1. Invite everyone in the department to share their thoughts on the situation with you in an informal or 20-min format. You can easily meet 8 people per day without jeopardizing your other schedule – thus, in a month, you’ll have met around 150 people.

2. Make a public pledge of transparency: you will keep all staff informed even if findings are difficult.

Step 1.2

Make a simple and honest maturity assessment across some key areas: requirement engineering & management, architecture, UX (not UI), dev process (individual dev environment, Continuous Integration tooling), testing, product documentation, Continuous Delivery & Deployment pipelines, and release management.

1. Within engineering, at 2 levels: management and seniors (in-person) and staff (anonymous poll, just ask for the team the respondent belongs to, within anonymity constraints so that individuals can’t be identified – remember people have lost trust at this stage).

2. Within adjacent functions: product management, customer support, professional services, cloud operations if separate from engineering – ask their leaders to fill in the survey in one of their team meetings.

The maturity survey scoring should be:

• Documented = 1

• Measured = 2

• Improved based on measure = 3

• None of the above = 0

In a struggling engineering org, many areas will likely score between 0.7 and 1.5, with some exceptions. What’s also very enlightening is the difference of perception from within engineering vs. within adjacent functions, highlighting silos, lack of cross-functional collaboration or emotional biases.

Step 1.3.

Complete the maturity assessment with an anonymous “cultural typology” survey at team level. You will be running it afterward every quarter or semester. This survey aims to find which type of culture the engineering staff perceives among these 3, as theorized by Ron Westrum whose work I mentioned in part 1. Here it is again for convenience:

The survey has 6 simple questions on a Likert scale (from 1 to 5 “strongly disagree – strongly agree”):

1. In my team, information is actively sought.

2. In my team, failures are learning opportunities, and messengers of them are not punished.

3. In my team, responsibilities are shared.

4. In my team, cross-functional collaboration is encouraged and rewarded.

5. In my team, failure causes enquiry.

6. In my team, new ideas are welcomed.

Insist on the “In my team”: respondents should not rate “the company” or “management”, which tends to happen instinctively. They probably have a different perception of the level “above” (e.g., their ideas and initiatives were regularly dismissed). This is ok and can be addressed later, after teams are strong, healthy, and delivering regularly.

I find this survey is far more enlightening for you and your management team than the yearly employee satisfaction survey (which is also useful, esp. for measuring eNPS, don’t take me wrong, but much harder to interpret).

Step 1.4.

As soon as you have gathered the responses, make a synthesis and organize an offsite with your management team (your direct reports) and a few well-selected senior engineers. Share the findings from above, and coach a problems/solutions brainstorming to confirm:

1. The issues in People, Process and Technology, and their acuteness,

2. The overall sequence of repair actions and (timing unit: a quarter), election of the first ones and how to undertake them.

Don’t be heavy with the change methodology. For short-term steps, I find OKR-like approaches work well and make progress transparent to the entire business, but you needn’t mention it is OKR or explain the OKR principles in detail (some engineers are a bit allergic to management consulting, shall we say 😊).

In short: stick to a no-nonsense approach.

Step 1.5.

Within 2 weeks following the offsite (important: don’t delay. You’ve created expectations and promised transparency!), call an engineering all-hands meeting, invite the adjacent functions (Product Management, Customer Support, Presales, Professional Services). Share the findings and kick off the journey.

If possible, make it an in-person meeting with a buffet lunch or afternoon snack at the end: informal conversations will spark, and you will get many more questions and interactions than the usual “any questions” silence you get at the end of an online event.

If you have multiple engineering sites in distant time zones, make the first one at the main/central site, invite some representatives from other sites, and prepare to travel: you must repeat this performance in person, even at locations with small headcounts. You want everyone on board and everyone to feel considered.

Step 1.6.

Set up a few key metrics and share them transparently but ONLY within engineering. The first one to start with is product quality. Yes, it’s a lagging metric. Yes, it’s an output metric, but you need to know because it’s leading to bad customer satisfaction, and since customers pay our salaries, everyone in engineering needs to understand how acute the product quality problem is (or isn’t !).

I suggest the following 6 metrics, which you can compound with appropriate weights (w=) and normalize them to a [0-100] “higher is better” range:

1. # of external defects per installation/tenant (w=4)

2. # of external change requests per installation/tenant (w=1)

3. Ratio of external change requests to total external records (w=1)

4. Age of external defects’ backlog (w=7)

5. Average severity of customer problems (use severity as defined in your SLA, w=7)

6. Trend of external defects (increasing or decreasing per period) – the most important one. (w=9)

Notes:

• External defect = found outside of engineering (by customers, support, etc.)

• Normalized score read: <40: RED, 40-80: yellow, >=80: green. A score below 60 spells occasional up to regular firefighting.

• Hint: an ok-quality product has under 150 total external defects per million lines of code. This has proven working for me for the past 20 years, independently of the product size, type, technology, etc. and correlates very well with the perceived product quality dimension in customer satisfaction surveys.

Step 1.7.

If your product quality index is RED, organize a commando operation to fix the ugliest: defects only, highest severity & oldest first – with judgment (# of customers likely impacted, customer revenue at stake). Limit this to 3 months. Force it through whichever planning cycle is used.

This seems somewhat arbitrary and forceful. Why is it necessary? If you’re in the red, the engineering team (and customer support) need some breathing room for the next steps, otherwise, you can’t hope to improve. Field reality will keep holding you back, like an elastic strip tied at your waist when starting your marathon.

Step 1.8.

If your department doesn’t have a minimal InfoSec practice in dev and ops, get an assessment of the situation as a matter of urgency: external pentest, external code & library review. Depending on the findings, prioritize repairs as part of the previous step: you don’t want to risk a security incident.

At this stage, based on your findings, you will have to explain to the leadership team that the current market release plans need to change and why. You will meet resistance, so make sure you have the CEO’s support. It’s really a sales act, where you will highlight the cost of doing nothing vs. the cost of the repair journey you’re planning. Here’s a metaphor I found useful: “We’re selling unmaintained cars, and our repair shop is a mess: customers will keep bringing their cars back, and we can’t hope to turn the situation around.”

If you’re not supported by the leadership team and your boss, you can’t hope to succeed in your transformation journey. Start looking for another job, I’m afraid it is as hard as that.

Once you’ve laid the foundations (diagnosis, communication, early trust), you’re ready to make visible progress. Let’s see how it could be done.

Stage 2: Detailed diagnostic and first repair actions (months 3 to 9)

This stage is about a deeper inquiry into previously identified issues and containment of the most urgent ones while planning the whole transformation journey. These steps are in no particular order, take what you need and reorder as you need.

Step 2.1.

Set up other metrics starting with DORA, these are the only research-proven dev productivity metrics I know of. Plan a session to explain them, share a good YT overview and encourage people to read about it.

Always compound several dimensions and always mix outcome & output metrics. In effect, no single metric, no single dimension can tell you objectively how (un)healthy your engineering performance is.

At the start, publish these metrics ONLY for engineering. Never let other functions access them at this stage: metric values will likely be low, and people without the proper background will likely misinterpret and possibly blame.

Step 2.2.

Review variable pay/bonus incentives. Change from individual to team-level, even if HR does not agree (you’re the CTO, you decide how your people are measured). Remove any output-related metrics in incentives – outcome only! It’s too easy for engineers to game output metrics. There’s plenty of literature to explain why / give examples. Trust me, I’ve made that mistake.

Step 2.3.

Ask your senior engineers to review the individual development setup and fix it fast if/as needed. I’ve seen teams where it took up to a week & experts to get a new dev started... You can imagine the impact on productivity, quality, and so on.

Step 2.4.

Review your CI and fix it if broken. Require it to be green at all times, it’s far more important than delivering +/- on time. A healthy CI is likely the only means you have to catch defects at this stage. If necessary, force it through in coming sprints and be nicely reckless 😊 about it.

Step 2.5.

Gather your senior staff & review key elements of the Definition of Ready and the Definition of Done. If anything is missing/weak: ask for an improvement plan, which they have to work on team level, for their team, and with all team members. Ask them to implement it and check personally that it’s not just on “paper”, and that it is measured.

Step 2.6.

Analyze the test strategy (likely, there’s none) & the test base at system level, i.e. the list of all system tests across the reference product functional structure (which is likely undocumented or incomplete)

• Looking for functional coverage: are there parts of the product which are not or insufficiently tested at system level? Check beyond the happy flow on critical features. Appreciate the % of automated system tests (likely in the 20%-40% range). Ask Customer Support to analyze defect density per main product function: you will know where to look and where to increase test effort.

• Patiently ignore opinionated senior engineers who will tell you that unit tests suffice to secure good product quality – it does not.

• Once you have this analysis, gather your leaders (seniors, managers) and help them build an improvement plan, focusing on 1) customer pain points with the product, 2) automation. Involve developers, not just testers. Actually, if automated testing is really low, everybody will code tests very soon, and at all levels (incl. system). Depending on your findings, ask for an improvement plan from every team, bottom-up, get them started, check their progress regularly, and celebrate it.

Step 2.7.

Depending on your findings in Step 1.8, and if your company doesn’t have an established product security practice, get one started. To a minimum, you need:

• A DevSecOps expert engineer reporting outside of your department (into the CISO at internal IT / SG&A is the usual choice), but make sure the person is paid from your budget so that you get them dedicated.

• If your product is SaaS, plan a certification, such as ISO 27001 or SOC2 Type 2, within the next 12 to 18 months. Start with the Ops scope, then Dev.

• An external pentest annually for all products and services, an internal one every 6 months

• A clear relation between the CVE score and the defect severity (CVE>=8 => highest severity defect (a.k.a. “Total Stop”) => immediate fix and release)

• Security by design training for all team leads, esp. on Threat Modeling. The DevSecOps expert will need to coach architects and team leads accordingly.

• Secure code as part of code reviews, and if you can afford it: a code scanner that can detect security vulnerabilities

• Update all outdated 3rd party libraries embedded in your product or planned replacement

At this stage, you’re ~9 months into the job (or journey), and you have identified your change agents, your champions – your golden circle.

Start meeting key/faithful customers and partners (VP+ level, esp. technical buying personas) and establish a personal, quarterly meeting routine with some of them. Recognize issues honestly (even if your sales VP or CRO doesn’t like it) and explain your plan. You can easily overlook this key step because you’re already with your head in the steering wheel, with growing expectations from everyone and high pressure, so be careful to keep a helicopter’s view. If you can’t see the whole engineering perspective, nobody else can. Don’t drown in customer crises, while they are very educational and need you to empathize.

Stage 3: Engaging in the real transformation journey (months 9 to 24-26)

This stage is about persistently grinding through change. First quick wins gave some hope to the crew, which supports their resilience while most of the difficulties are still ahead.

Step 3.1.

Gather your senior engineers & architects and ask them to assess the technical debt by nature and ask for an improvement roadmap. Hint: it’s a 2-year-plus roadmap. The actual journey will likely be somewhat different down the road; what’s important is that all get a picture that makes sense (i.e., why certain things need repair before others).

Step 3.2.

In good cooperation with your colleague CPO, set up guardrails (yes, I know, guardrails come from Scaled Agile, and SAFe is challenged – but there are good bits in SAFe 🙂) for quality, refactoring, AND learning. From this point on, make sure each iteration has its dose of bugfix & refactoring, as needed, but don’t be dogmatic on percentages and don’t make it a “one size fits all”.

Make sure managers check that every engineer has at least 2h per week for personal learning on the job, and check it’s not a “fake” commitment, for example by fostering prototyping initiatives or spikes inside the sprint planning, which also have the benefit that people will structurally feed their findings back to other team members and beyond.

Launch a “lunch & learn” routine if you have enough volunteers, and include yourself in it. In between 2 iterations, create space for half a week to a week of hackathon for all teams and accept ideas even unrelated to the business or the products.

For refactoring: make sure it is done by (very) small increments while you steer your organization towards continuous delivery. Don’t allow branching out for longer than 1 iteration – read here for further information.

A good metaphor to picture the refactoring journey: the renovation of historical buildings. While the façade and key artwork is preserved, all the rest is progressively and completely re-hauled to modern standards. A product isn’t a disposable prototype. It must be treated like a heritage site: valuable, functional, and worth preserving through gradual renewal.

Step 3.3.

Engaging culture changes. You want to foster Autonomy, Mastery and Purpose for every single engineer and team. You want to bring your organization into a generative culture, as Ron Westrum defines it.

Review your senior staff & the management chain. Don’t be harsh, don’t punish, but fix the obvious casting mistakes quickly (poor leadership for example, or people who have been kept in the shadows, especially women).

Help people to revert to a position they can succeed, or if it’s not possible, manage exit professionally (it’s rarely the person; it’s almost always the person in the context). For example, team leads lacking empathy or impact are often excellent technical experts, well-recognized as such by others. Another example: a colleague might be underperforming just because your organization is too chaotic, and the same person would thrive in a more mature organization (or vice versa).

Step 3.4.

Prepare a dual ladder and a basic talent management detection process for engineering with the help of HR. Both don’t have to span the entire company, but you need to have them soon for engineering, or you’ll lose your best people. Prepare a pay gap analysis for the next round of merit increase: you will probably have to use a good part of your merit increase budget to fix discrepancies, and thus can take 2 or 3 years.

Step 3.5.

Review and get the technical, team-level onboarding routine fixed, with clear objectives (again, you can use a lightweight OKR approach). For example: “In 3 months from now, newcomers’ technical onboarding will be down from a week to half a day”. Or: “After 3 months of tenure, each engineer can pitch the product value and demonstrate the product to a customer”.

Create a schedule so that every single engineer is certified on the product at a “pro” level, using resources from your customer training organization. Partner with sales engineering & professional services to achieve this.

Step 3.6.

Create a “live my life” program with Customer Support, so that each engineer starts getting closer to customers. For example, start with 1 “discovery” day, then set up a “1 week per quarter” rotation. Consider replicating it with Professional Services and Presales, but then voluntarily. With product management, secure regular engineers’ participation in customer visits, and make sure every team gets its dose.

Step 3.7.

Review the teams’ organization, team members, and change what needs to change. Be careful not to go too hard or too fast, and keep away from dogmatism such as “let’s have feature teams, let’s have platform teams, etc.”. Customer context and product context (f.e. product architecture) are your guides.

And when it comes to forming teams, quality is in the mix! Mix juniors, mediors, and seniors. Mix men & women. Mix nationals and colleagues of foreign origin. Don’t worry too much about hard skills consistency and be reasonable with time zones. Nature, with mixing genes, is your foremost example.

Final advice for the CTO

That’s about as far as I can go in detailing the journey. Let me now end with a few pieces of advice for you, the CTO:

• Be radically transparent—but careful in tone.

• You’re a coach; don’t do all the talking

• Don’t rush the “how.” Let your teams own the tactics.

• Stay connected to technology. Your judgment depends on it.

• Manage expectations clearly with everyone in the business

• Make yourself accessible, max meeting time = 50%

• Read Accelerate by Nicole Forsgren & al.. It is a MUST READ for C-level in software engineering

• Prepare to be resilient yourself...

Finally, remember that your goal is that your organization can work “without you” (so to speak 😉) in 3 years’ time. I’d even say: your goal is to make yourself redundant and prepare your successor to take your position.

If this resonates or you’ve walked this path yourself, please share your thoughts. If you’re struggling in such a journey, reach out for advice or consulting.

I wish you all a fruitful journey!

This article is somewhat technical as opposed to Part 1. I’ve made a sincere effort to make it digestible for non-technical audiences. In effect, I hope it speaks to non-technical audiences, especially decision-makers, with the hope that it will shed light on some “techy buzzwords” they’ve repeatedly heard, without getting clear examples of what they relate to (“Technical Debt”, as an example).

Process issues

Software engineering is complex, so it seems important that it is done the right way. Let’s open this next Pandora’s box.

Poor / ad-hoc software testing process.

Agile Principle #7 states:

So, we better know it’s really working.

Software testing is complex because of the huge combinatory explosion of inter-related parameters, conditions, and functions. Since you can’t hope to fully test a piece of software, how do you decide what’s “enough”?

• How do you know at which level (unit, integration, system) you need to focus your test effort, and by how much?

• How do you know you’ve covered the highest-risk areas?

• How do you know that your tests are disjointed, i.e., that you’re not testing the same feature several times?

Next to that, testing quality attributes (performance, usability, maintainability, ...) can be very complicated, few engineers know how to approach this right.

Unfortunately, software testing seems to be the software engineering’s poor man:

• It has a bad vibe from the past (historically, testing was manual, and referred to as “monkey testing”)

• Testers are chronically less paid than developers, but since tests are becoming more and more automated, they actually do the same job.

• Software testing is a “by the way” detail in university curricula rather than a discipline of its own right.

• Software testing rarely offers high & well-paid expertise positions (have you ever met a distinguished or principal test engineer?). Luckily, with Site Reliability Engineering (SRE), extending QA automation, this is starting to change.

• Software testing is fraught with clichés. “If you have more than 90% unit & integration test coverage, you don’t need system testing.” 🙄.

To add to that, business leaders explain that “you can’t sell quality” (or refactoring, a key ingredient to sustaining better software quality) like you can sell a feature (but who sells features today?!). You know, the same business leaders who keep stressing the importance of NPS every single day.

Consequences of a weak software testing process: well, I just mentioned it: low customer satisfaction in the product dimension, leading to low NPS. Compare it to flight safety procedures: even a single overlooked check can lead to disaster. For executives: If your engineering department isn’t advocating for software testing, ask why. Are they lacking the right resources or leadership support?

Customer feedback through proxies.

Agile Principle #1 (yes, the very first one, on which the entire Agile philosophy is founded!) says:

If engineers can’t talk directly to customers, how would they know how satisfied customers are, and by what? Is it a wonder that start-ups cultivate this to the highest degree, as a matter of survival? But then: why is it that direct engineering-customer contact is one of the first things that goes away when the startup scales?

In most of my career, I have rarely seen engineering teams with direct, regular contact with end-users. They don’t sit at Customer Advisory Boards - business does. They can’t afford user interviews / User Scene Investigation techniques. They don’t get a budget to implement telemetry in the product. The few cases where this happened, typically in small startups or research-driven teams, demonstrated how powerful this practice can be.

Yet, all these organizations claimed to be Agile... My personal research and reading tend to confirm that it is not uncommon. How can this fallacy persist?

I understand that often, end-users or customers make engineers uncomfortable. They’re often introverts, or fear facing difficult customers’ questions. They know they can’t make any commitments when challenged with functional or quality issues. They may fear Sales will skin them if they lengthen a sales cycle. Or they’re simply forbidden to speak with customers because their management is concerned with leaking future Intellectual Property, or because another function (Product Management, Sales, Marketing) claims customer contact is their prerogative.

Sometimes, let’s be honest, it’s engineers themselves who don’t care: I’ve witnessed a few saying “I’m here to code and solve complex technical problems, I don’t care about the rest” (value, product, domain, users, customers, you name it), this being a “not my job” call, without management or leaders correcting. Hey, folks, wake up! Customers pay our salaries! How do you feel when you hit a bug in your favorite game and support does not answer?

Consequences: lack of Purpose, leading to lack of motivation. Features are not slick, to the end-users’ frustration. Poor service to customers. All leading to the loss of new business or customer attrition, to the frustration of Sales and Customer Success, who don’t even realize that the root cause is the violation of Agile Principle #1.

Process dogmatism, one size fits all, process over-engineering.

Beyond missing customer feedback, software teams often suffer from rigid processes that fail to adapt to their reality. Let’s explore how process dogmatism hurts engineering teams.

“Let’s implement Scrum” (or SAFe, another frequent example). Why?! Why not Kanban, Scrumban, XP...?

“Everybody does TDD, let’s goooo!” Why? Is it really necessary? Is it even doable given the technical debt?

Why don’t you experiment, try at a small scale, and learn? Trust teams will find what works best for them to achieve the desired outcomes. (Agile principle #5)

“I require 90% code coverage at branch level in every team.” Why? Is this risk-based testing in any way? Is it even doable given the technical debt? Besides, proper test beds and especially mocks can be very tricky and time-consuming to set up. What about legacy code: how can we hope to catch up to 90%, while it’s in legacy code that the current issues lie?

Let’s remind ourselves of Agile Principle #12:

A common pitfall is to ignore what teams advise for themselves in the development process. Managers: you don’t do their jobs! (or not anymore). While managers often have prior hands-on experience, they don’t necessarily know better. Do you think Scrum can work in a team swamped with support tickets? I don’t think so. If you force the development process, here are some consequences you will quickly get: loss of purpose, loss of motivation and morale, energy spent complaining after bosses at the coffee machine, loss of productivity, and employee attrition.

Excess or absence of process support

I’m referring to chores carried out by job roles such as scrum masters, PMOs, or tooling support staff (a.k.a. platform teams in today’s lingo – life is a journey of re-discovery it seems 😉). Or product owners, for certain aspects of their work.

I’ve seen many Scrum teams where ceremonies become bureaucracy: 1h-long standups, poor moderation (discussing how to fix issues instead of just listing, confirming, and scoring them), iteration planning sessions lasting more than 3 hours, and involving all teams & members including those not concerned in any way with parts of the conversation, sprint demos & retros longer than 1h, inspect and adapt while there’s nothing special to inspect, or adapt actions remaining wishful thinking, user stories going in far too many details or, worse, tackling the “how it should be engineered”, etc.

Conversely, I’ve seen teams completely let down with tooling, quality input data, and information maintenance. Poor user stories (e.g. acceptance criteria unclear or not focused on user outcome), stories only addressing the happy flow or ignoring error conditions, fat backlogs that became paleolithic archives more than anything else, lack of templates for ceremonies, decisions not recorded, product owners responsible for too many teams and unable to answer clarification questions on the spot, team leads coding 100% of the time instead of supporting and coaching other team members, test cases disconnected from user stories because nobody checks…

Consequences: engineers complaining about bureaucracy and occasionally using this as an excuse for missed objectives, engineers ranting about “non-productive” headcount esp. when the team is missing positions. Negative energy (or loss of positive energy). Broken team dynamics (loss of communication and trust between team members). Loss of productivity.

Lack, excess of, or flawed metrics.

Recent research (cf. DORA metrics) makes clear that appreciating software development productivity requires careful co-interpretation of specific sets of metrics. However, many leaders (including myself until recently, I’ll be honest) keep focusing on one or a few disconnected ones. Other typical pitfalls: detailed output metrics and no outcome metrics, metrics to please upper management instead of helping teams to improve, and metrics just for variable pay.

Heavy-handed metrics twist engineers’ behavior: they will work for the metric and not for the customers’ benefit. They will game the metrics (famous example: managers asked to increase velocity -> teams redefine the “value” of story points or cut stories into smaller bits) and managers will not notice.

Consequences: loss of Purpose. Demotivation. Staff attrition, losing the best engineers.

No room to experiment or innovate.

Or very scarcely and only for the happy (?) few. No spikes or genuine feasibility studies for complex implementations.

I’ve already addressed this somewhat in Part 1 of this series, so I’ll keep it short: when the engineering process itself does not support learning on the job, you drastically reduce the chances of discoveries, value improvements, new product ideas, or overcoming certain parts of the technical debt.

Next to negative consequences on individual and team spirit, you risk losing quality and future business opportunities.

Process issues hampering good engineering are not just in engineering though. They can come from adjacent departments. Examples: pre-sales or professional services twisting the product beyond its specs, for example using product internals to configure a customer instance or disclosing internal APIs to customers. This, to me, shows a lack of quality control at their level, while the product and the engineering team can hardly do anything about it (other than keeping their encryption tokens safe and non-generic). Another example: the same departments refusing to participate in end-to-end testing, or even not opening defects against the product, while their field knowledge could provide invaluable insights (”not my job”, silos...).

The art of process is not just to understand the theory, but also and mostly to be able to use it and apply it in context - take what’s useful, simplify it as needed, and ignore the rest. You don’t even have to name them!

It’s not à la carte though, the process needs to be coherent and comprehensive.

Technology issues

Let’s now review the last pillar of the “holy trinity” of People, Process, and Technology in some detail.

Technical Debt

Let’s start with the elephant in the room, the arcane and fabled technical debt. Compare it to deferred home maintenance: you must regularly repair or improve certain parts, some more frequently than others. Small ignored issues become costly disasters.

Examples: spaghetti architecture, prototypes that became part of the product due to time-to-market pressure. Rushed features from the early days when grabbing business was a matter of survival. Outdated technologies that don’t scale. Vendor lock-in putting product costs at risk. Cloud costs exploding because your technical stack scales linearly with users / tenants. And so on.

Specific to M&A: an overly diverse technology stack, consequent to several acquisitions. Rationalization has not been planned and budgeted in the post-merger business plans.

Technical debt is inevitable: new technologies and new architecture patterns emerge constantly while evolving usage conditions make those we have in our products age, sometimes quite fast. We must live with it, anticipate it, and embrace it.

It becomes a major impediment to business growth and customer NPS when it has not been consistently and regularly addressed through what I call “patrimonial maintenance” – to the point it may simply limit the company’s growth horizon. The more you delay addressing it, the longer you’ll pay it, and the higher the interest on debt. In a way, it’s like public debt in some countries: you continuously borrow just to repay interest, and the principal keeps growing. And your customers are the ones paying for it without you appreciating it, and at some point, they may deny you further credit.

Do you know that it suffices 10mn research on the likes of Glassdoor and Trustpilot to get a good clue of what your technical debt looks like? Comments like “old technology” or “good product but difficult to configure” are very good hints.

Consequence: low customer satisfaction with the product, leading to NPS dropping, loss of reputation both as vendor and employer, and customer attrition. The company will progressively lose market value. It is slow, almost imperceptible in the beginning. Company leaders don’t perceive it; it’s insidious. Year-on-year growth rates slowly decrease from the glorious 20+% to below-market CAGR. Until one day comes an acquisition offer from a bigger / better competitor, at a disappointing price point. They will likely grab the market share, kill the original product(s), and eliminate redundancies.

Security debt - both on the product and on the tooling.

Product: 3rd party packages are not updated regularly. Lack of threat modeling on new features. Infrequent or no external pentesting. No review of legacy code from a DevSec point of view. No DevSec training for team leads and beyond.

Tooling: CI elements not updated, or not changed while proven to be major security “holes” for supply chain attacks. And beyond: no or flaky BCP / DRP, or not tested regularly.

Consequences: reputational damage at the first security incident. Loss of customers. Loss of market value... Get yourself an ISO27001 / SOC 2 Type II certification fast, both on Prod and Dev.

Lack of proper technical documentation

Engineers must discover how the product works internally by looking at the code. Over a codebase that’s typically several 100.000s or millions of lines, you can imagine what the learning curve is. Over time, original engineers left for various reasons: better pay, more responsibilities, and out of frustration to experience a better / more state-of-the-art technical working environment. Those who remain are hardly coding anymore because they spend their time helping others understand how it’s built. Or they simply don’t care, do their job, and help nobody because nobody helps them get out of this ditch. Do I need to spell out the consequences? I guess not.

Lack of representative and comprehensive “real-life-like” test data for end-to-end testing.

You might be surprised: shouldn’t it sit in the Process section? I claim such test datasets are a technical asset. They can take a lot of work to generate and require solid field knowledge (which is scarce). Often, customers won’t agree to use theirs beyond a support ticket, even if properly obfuscated.

Also, when your software drives target hardware, these can be very expensive to obtain or difficult to get access to.

Yet, if you forgo this effort of securing your real-life test data or environment, you won’t be able to complete your test strategy, and complex field defects will bite you back, resulting in dissatisfaction, support overhead, flying staff to debug at customers’ site, and so on.

Poor tooling, broken CI pipelines, lack of standard development environments

I’ve mentioned it already in Part 1: a good onboarding experience strongly correlates with a company’s business performance – there are many studies about it. In IT, because human resources have been and is continuously scarce, HR and hiring managers take care of creating a good first experience and do a good job with employee onboarding at company level.

However, for various reasons, poor technical onboarding is not uncommon. Clunky individual development environments do exist, CIs with many workflow exceptions and manual bridges are not uncommon. This creates an eerie feeling of “what’s going on here, what have I stepped into?” sentiment with newcomers. The sad thing is, after some months, people (especially juniors and mediors) get used to it to the point that they forget how counter-productive such an experience is, and nothing improves.

Unconsciously, it becomes part of the norm. You don’t notice it anymore, it’s like a painting on the wall that you’ve always seen, and to which you don’t pay attention anymore.

Specific to M&A: changing the project management tooling. Imagine your department works with Jira, and you’ve just acquired a company where people work with AzureDevOps, Redmine, etc. If you want to realize the planned technical synergies (joint use-cases, shared componentry, unified platform strategy, synchronous delivery), you’ll have to force one side (in general, the acquired one) to move to the other. I’m not saying this is a mistake, and there could be situations where you won’t have to do it. But such a move is structurally complex and will likely disrupt engineering for a month or more.

Luckily, all these tools offer comparable features and experience, with marginal pros & cons. Many engineers already experienced different ones with different employers. The learning curve is not too steep. Yet, this alignment of tooling will meet resistance and some loss of momentum and productivity.

Technology fancies

Like the Process dogmatism I was writing about above, some manager or senior engineer with authority went to a conference, read books, and watched some YT videos about recent shiny technology and started actively lobbying to try it.

“Let’s do microservices! Let’s do serverless!”, etc. Likewise, Product Management comes with “Let’s put AI in the product” without any clear use case.

Reasonable engineers are starting to ask “why?” and often get no better answer than “because everybody goes for it and has super-duper benefits (on paper...) and it’s the future”.

Man, oh, man! I’ve seen many products, where a solid and well-engineered client-server monolithic architecture was doing the job and was able to scale for the next 10+ years, go into the ditch because of such initiatives. Bits of Docker embedded into architectures which were never ever designed to sustain it. Event-driven architecture initiatives, that make the product quickly very tricky to diagnose and maintain because the original logging was neither designed for it nor properly adapted.

A small, supposedly positive, and future-oriented initiative can result in a support nightmare and furious customers 3 years down the road and will take another 3 years to fix. I remember a company having developed its own event broker service back in the day because open-source solutions were not robust or fast enough, and commercial solutions outrageously expensive. The company nearly went bankrupt because of that.

Consequences: the best engineers leave and sell their vested stock before it’s too late, and you must cope with the resulting mess with underqualified staff, who are all the more frustrated that they did not create the problem in the first place. Loss of productivity, customer satisfaction down the drain, loss of NPS. Hostile takeover by a competitor…

Bottom line, including Part 1

There are many types and sources of People issues in software engineering. These influence strongly and negatively the ways of working (Process), and the perennity of the product’s Technology internals and tooling – and the other way around. All the while, the company is rushing to make the numbers, with little or insufficient attention to these unpleasant, “techy” matters. I am convinced that it’s worth taking a step back and acknowledging these hurdles.

As always, comments are welcome! Did I forget an obvious type of impediment? Any crushing anecdote you’d like to share (if you’re concerned about confidentiality, I can proxy à la The Register with their excellent “Who, me?” articles – just contact me by PM). Software engineering is so complicated that there can’t be enough experience-sharing.

These challenges are deeply ingrained, but they can be addressed. But there is light at the end of the tunnel, and every problem has a solution. In Part 3, I’m mapping a pragmatic journey to overcome these hurdles.

Before I start, let me clarify the setup considered here: if you’re leading a software engineering department with 50+ engineers and find yourself constantly firefighting rather than building, this series is for you.

This article, part 1, is summing up the issues I’ve faced in the People dimension. Part 2 will summarize issues in the Process and Technology dimension, and the last one will propose an approach to overcome the issues I describe in the first two.

In the second half of my career, roughly from 2009 onwards after I reached D-level, I’ve spent much more energy trying to understand and fix failing software engineering than actually building great and innovative solutions - to the point that it has now become my main professional value proposition.

It’s not just me: patterns which make engineering teams struggle seem remarkably consistent across the industry. How is that so?!

Don’t take me wrong: I don’t mind addressing these challenges at all, quite the opposite. Improving or fixing complex systems in all dimensions of People, Process and Technology is an integral part of what we, as technology leaders, are trained for. We love it, and it is very rewarding.

But to be fair: I wish I had more occasions to start from a white sheet and have it first-time right, or close to, like those folks who started at some big names like FB or Uber in their early years.

But this is not about regrets (which as we all know are useless), it is about summing up what I’ve learned in this Bob-the-mechanic journey - most importantly with the hope that some of you readers will either avoid mistakes I’ve seen or made or are better equipped than I was when facing dysfunctional software engineering.

People issues

Software engineering is difficult to get right because each dimension of People, Process & Technology is complex. A well-oiled software engineering department requires delicate balancing acts, requires all 3 dimensions to be “healthy” so that it ultimately delivers profit for the company. While I don’t have personal experience in other engineering verticals such as manufacturing or construction, I don’t know any single one where such a level of complexity shoots so early in the company’s history.

I claim that the most complex part, the most difficult to get right, the most delicate to keep right, is the People dimension. I’ve learned it the hard way in every single experience I had, including those where Process or Technology were ok.

Dysfunctional team dynamics are at the heart of this People-related complexity. I’ll describe my take on the key issues in 3 parts:

1. Through the lens of the HR System

2. Zooming on Engineering Management

3. Taking the broader Cultural perspective

Issues throughout the HR System

Let’s go chronologically through the main domains of the HR System.

Recruitment mistakes or tradeoffs

It is difficult to pick the right person, despite all the tooling and methods we have for screening candidates. It’s a gambit, and the probation period is there to help us check facts.

Recruitment mistakes are typically related to skills (especially “soft” ones, such as leadership, or can-do attitude, or abstract thinking…) and fit. IT in general, and software engineering in particular, has been chronically under heavy resourcing pressure. Remember Y2K, move to Euro, Mobile, then Cloud, Data Science, now AI or Cybersecurity: we often had to hire under-qualified candidates out of necessity.

These resourcing issues lead to loss of productivity, with high overhead for team leaders and seniors.

Poor / no technical onboarding

There are many research papers correlating poor onboarding to high turnover and low productivity. My point here is specifically about getting new engineers up to speed with the team’s working environment. No technical buddy, little or no documentation on tooling, product architecture, coding standards, testing. No planned hands-on product training. This increases ramp-up times, inadequate quality from newcomers’ early contributions – even while their past track record is good.

Basic performance management for engineers

I’m assuming the company already has a proper (even simple) HRIS. I’m referring to symptoms like:

• Managers not knowing how to set SMART objectives – in software engineering, the Measurable component can be especially failing.

• Managers not trained to appreciate the behavioral component in performance.

• Individual objectives only, no team objectives

• Objectives on output metrics instead of outcome metrics.

This leads to frustration, jealousy, lack of commitment, lack of ownership, engineers gaming the system, with negative consequences on productivity and quality.

No talent management

It’s not just about performance; it is also potential that maps the career journey. Are people Insight Focused? People Orientated? Change Motivated? Result Orientated? Do they Aspire to grow?

And with lack of talent management: no job architecture, no career planning, no clear evolution or growth perspectives, no dual ladder (leading the best technical performers to the management track, which they didn’t ask for, in order to earn more).

Next to the previously mentioned consequences, this leads to losing your best people.

Poor Learning & Development experience

Classic (and boring) classroom training, with no follow-up on application: the learning sticks only if you practice it right after your training. A specific example of that: old-school management training, and no leadership education for senior staff.

No budget to go to technical conferences. And no genuine time to learn on the job. Managers say: “Just do it in your free time at home” - but every engineer already does that!

Engineers are passionate about learning new things, and the opportunity to learn is a key motivation factor to join or to stay in a company.

Consequences? Already listed, and I won’t mention them further below.

Flawed promotions

Here’s some I’ve seen chronically:

• Managers promoting their “pets.”

• Giving senior or higher status to people with weak or no leadership, empathy, or genuine interest in others

• Promotion to management on the sole basis of technical seniority, as a means to give a pay raise to strong individual contributors.

Pay inequality that cannot be explained.

• Lack of transparent job architecture (e.g., what does it take to become a senior)

• Pay gap between new joiners and engaged employees with a long tenure (10% to 20% is common.

• Gender pay gap – a proper scandal.

If there’s ONE job category where pay is benchmarked, it is software engineering. Everyone knows pay ranges per specialty and level in their region. There are plenty of public sources available. Besides, after some time in the company, everyone has a good clue about their colleagues’ pay.

Repairing this takes 2 to 3 rounds of annual merit cycles – and since you get a fixed merit increase budget, you must do it at the expense of the rest of the staff – who has no responsibility in the issue.

Poor exit management

No exit interviews, or not shared with managers, or not synthesized into corrective actions.

Luckily, I think this is going away.

That’s as much as HR management is concerned. The next people-related issue I want to discuss is:

Broken engineering management

Management is a discipline of its own right. You cannot improvise. Let me list a few common mistakes, some of them I reckon are not specific to software engineering.

Not answering “why?” Why this feature over another, why features over quality & refactoring, etc.

Purpose is a key motivator in everyone’s life, including on the job. Engineers are particularly good at asking this question, at challenging the answer (it’s not a game: we really need to know why to figure out the “what” and the “how “to do a good job!).

Engineers are very analytical. If the answer does not make sense to them, you’ll get lack of ownership, lack of commitment, low productivity.

Failing to empower teams. Lack of listening, understanding and trust does not yield good performance, while shared leadership does. Also, challenges from managers of adjacent functions, who influence de facto engineering, can have an adverse role.

Framing individuals and teams in the iron triangle. Hard-committed delivery dates with fixed resources and content. Since the 1950s, we know that it does not work. If there are no degrees of freedom in this triangle, quality will be poor. It’s inevitable. Yet, some managers persist or can’t negotiate tradeoffs while representing their teams.

Shielding software engineers from field reality (or vice-versa). A typical example is forbidding customer support reps to talk to engineering, “because you see, engineers need a serene work environment to do their complicated job, so let’s not disturb them”. But they need to know, because only they can fix it, and they’ve created it in the first place (while not intentionally, of course)

Or the other way around: bombarding engineering teams with the latest support issue or the last idea of some big customer (committed under the hood by some sales leader. Disrupted planning leads to more pressure, more delay, less quality, more technical debt, resignations.

Ignoring human dynamics when forming teams: social styles, personalities, seniority levels, diversity. Do you know how easy it is to manage a team of young alpha males who want to be architects, seniors, managers, etc., while they don’t qualify at all?

Lack of managerial courage. For example, delayed structural changes, then sudden organizational, process, or technical revolutions. Inability to address underperformance properly and professionally leads to managers losing credibility and trust from other team members.

There’s a great saying in Dutch: trust arrives on foot and leaves on horseback (vertrouwen komt te voet en gaat te paard). You see my point.

Weak change management, especially not listening to feedback on changes from staff & teams, not involving them in why, what, and how to change, expecting change will happen on top of the existing workload automagically.

Rants on Work from Home, while software engineering is THE job which is most WFH-friendly. Don’t take me wrong: I’m not a proponent of full remote, and WFH must be organized so that team members have an in-person routine and can also meet in person with other teams, as necessary.

In-person work is key to build trust in teams, reinforce a positive engineering culture, smooth out disagreements or conflict. Besides, certain engineering chores are done more efficiently in person (creativity or planning sessions as an example). In-person rotations are not difficult to organize, many companies have done it successfully.

On the other hand: software engineering requires extended periods of uninterrupted thinking, especially with complex architectures. Why would engineers burn time commuting to sit alone at their desk, chatting on Slack with a colleague two desks away? I honestly don’t know.

Nah – I believe these rants are essentially coming from leaders with excessive control anxiety.

Micromanagement over coaching from former software engineers who’ve been promoted to first line management. They have patterns without reasonable “whys”: control (“I’m the boss, do it as I say”), assertiveness (“I know better”), technical hobbies (”let’s have a microservice”).

Yourself as a CTO! Yes, you’re part of the problem when you don’t support your teams and directs, often by lack of attention to their issues, which you disguise for lack of time. You create the problem when you are unable to say “no” to the business when needed or negotiate tradeoffs. You’re part of the problem when you overpromise and your organization under-delivers. Consequences: loss of trust, loss of motivation, etc. - and possibly yourself losing your job.

Broken (engineering, but not only) culture.

Dr Ron Westrum, an American sociologist, has published key research in the area in 1988 based on his experience working for the US military. His findings apply fully to software engineering. His system lists three types of cultures with their characteristics:

Looking at the first 2 columns: how many of such organizational cultures have you seen or worked in during your career? Compare to the third. My personal experience: 10 to 1.

Even if you have well educated and benevolent engineers, even if you have the best toolset, even if you use the greatest technologies: I believe you can’t hope to have a performing engineering department without a generative culture. It’s true for all functions and all types of businesses, of course, I’m just saying that it is acutely true in software engineering. It heavily relies on brainpower, and good brainpower only comes when there’s peace of mind.

Note lastly that engineers themselves, hopefully without knowing, do contribute to the problem. Urgent support questions sitting an entire day in Slack. Unanswered questions from another team that has dependencies on theirs, etc. And their managers often don’t see the issue or ignore it, only a few senior “control tower” individuals maintain bridges the best they can.

I appreciate that this list of misery is depressing. Yet, it’s reality more often than not, so we need to responsibly acknowledge and embrace these issues. I also want to be clear that I’m no saint: I’ve made many of these mistakes, and some of them quite recently.

Let’s conclude this first part on an optimistic note by reminding ourselves of the 5th Agile Principle:

Continued in Part 2: Process & Technology.

At some point in its journey, a startup will meet success with their first product. As revenues and profit grow, the company will need to decide where to invest next in product development. Several options are available:

1. Additional functionality in the flagship (paid option or not) or improving key quality attributes (performance, scalability, user experience - to name a few). For example: adding scripting or macro capabilities like Microsoft did with VBA in Office applications. This could also be a mobile client, or a SaaS version, or a companion product which requires the flagship, like a remote access client.

2. Pivoting your flagship to address a new customer category: from Corporate to Small and Medium Business, or vice versa, or adapting to new verticals. This is quite challenging: certain requirements are very different (think: ease of use, stability, or interoperability), while the need & value proposition are similar / the same.

3. A new product in the same functional domain. For example, Atlassian introduced Confluence after they had launched Jira - both addressing project management in software engineering, one from the planning & execution angle, the other from the knowledge and documentation angle, and complementing each other.

4. A new product in another functional domain, often adjacent, using the flagship’s existing technology to expand into a related market. For example: applying computer vision (image recognition) to OCR (Optical Character Recognition), enabling new use cases in forms processing.

The discussion I’m proposing today is: which should you choose, and why? How can you secure the Return Over Innovation Investment (ROI2)? Let’s review these options.

Option 1: flagship functional augmentation

The crux: customer retention vs. customer acquisition

In the early stages of the flagship’s lifecycle, the user community will actively request additional functionality or improvements, and the business will have to decide whether such efforts, next to keeping the customer base happy, will be compensated by accelerated new sales. Strengthening customer relationships improves NPS, which is linked to faster sales growth, but it is also likely that new-new sales will continue progressing without major development efforts.

Here are a few questions that could help:

• Have you saturated your current geography & verticals? Do you really need major innovation in the product to expand to adjacent geographies or verticals?

If the answer is “probably not”, you should invest in polishing the flagship (a.k.a. “patrimonial maintenance”) and invest in several new product PoCs to prepare for your next 10 million ARR.

Another way of looking at it: as long as the flagship has not crossed the median of the innovation adoption curve and your product quality/experience is good, you should not over-invest in the flagship.

• Is your flagship was a precursor in the market, or is it a late entrant with prior and established competitors?

If the former, honestly, you should be very cautious not over-investing in the flagship. In the latter case, you will want to increase the flagship’s market share. A thorough House of Quality benchmark (hands-on evaluation of competitor products by your Subject Matter Experts) together with a high-quality Voice of Customer approach will help you determine the product areas where you want to stay ahead, those where you need to catch-up and those you can sacrifice.

Be cautious with market analysts who push trends in a specific direction. While they can identify emerging needs, their adoption timing is often optimistic. If they tell you: “this feature will be the next big thing in 3 years”, you can safely assume it will be 5-6 years instead.

• If you reduce your engineering effort on the flagship to bare maintenance/bug fixing, will you really miss sales?

I appreciate that this question is quite provocative... but I think the answer is more often “no / not really in the short term” than not, while your standing organization will try to preserve their budgets. With our heads in the steering wheel and our horizon set to the end of the fiscal year, we tend to repeat what we’ve always done - this was successful so far, and we’re making good profit, so why change?

We’re gregarious animals. We tend to stick to what we know and what works. This makes us safe.

If you can objectively answer “we won’t miss sales in the next 12 months if we stop engineering”, then this is another sign that you should pursue options 2, 3 or 4.

Before discussing these, I want to stress two very important preconditions: your flagship needs to have a spotless product quality, and a low level of technical debt. Otherwise, your engineering capacity will not be fully available to build the future, and recurring customer support issues will generate adverse tensions in your organization.

Option 2: pivoting the flagship to address a new customer size segment.

I won’t discuss this one extensively. The key requirement here is that your software architecture, especially at platform level, must be:

• Flexible enough to support a significantly different user experience (either much simpler for small customers, or feature-rich for bigger ones)

• Capable of supporting a different set of technical requirements, esp. robustness, stability, and scalability

If that’s not the case: forget about it, or fold it into your investment P&L, unless poor customer satisfaction will bite you back hard.

Option 3 or 4: a new product and/or a new market

You made the clear diagnostic that it was time to launch a new product. New-new sales growth subtly slows down, while your sales organization is the same, and performing. Customer demographics show you’ve reached over 50% of your target addressable market and you’re an established regional player (going global is yet another challenge I’m not discussing here).

I’ve often observed that software companies tend to rely too heavily on their first successful product, to the point that the organization struggles to think beyond the flagship, even more so if you have delayed starting a new product journey. In particular, sales tend to stick to what they know is successful.

At the beginning of this new journey:

• The product and engineering organization struggles to identify a compelling enough value proposition for the new product (they keep thinking “in the box”). Subject Matter Experts keep spinning the same record. The Voice of Customers keeps advocating for the next version of your Porsche 911.

Resulting in:

• 6 months after launch, even with strong sales enablement and marketing campaigns, the new product struggles to gain traction. Your sales organization keeps reporting missing features with the flagship, keeps explaining why your new product doesn’t work instead of suggesting solutions, special sales incentives on the new product sales have no effect, and customers ignore your new baby. The few sales you get are from junior sales reps or recent joiners.

What I’m hinting at here is that there’s a certain likelihood that your Revenue organization is your biggest challenge. The bulk of the effort will not be with engineering, but with changing the mindsets, and this needs careful planning and thorough investment.

I’ve seen many business cases where this is not accounted for at all: investment P&Ls focus on engineering costs and ignore completely the necessary investment into sales and marketing, assuming that the customer-facing standing organization will “just do it”.

You need a fresh perspective. The original founders’ vision, that something important was missing for a well-identified target market, is quite hard to reproduce in an established environment — and this often includes the founders themselves. Coincidentally, it is often at this stage that they sell their stakes to move to their next entrepreneurial challenge.

A fresh perspective is easier brought in by adding “fresh blood” to your organization (new joiners with a strong motivation to succeed, in particular new Subject Matter Experts, Technologists, and a small, dedicated, and entrepreneur-minded sales group) together with selected existing employees willing to take the risk. You will need to protect them in the beginning, until they have demonstrated to the rest of the organization that it actually works. Another avenue is M&A, which de facto brings this new perspective on the market, while it comes with its own challenges, including differences in company culture — but that’s for another discussion.

To sum it up:

• If you can reduce your engineering effort on your flagship product down to maintenance with little consequence on sales and customer satisfaction, it’s a sure sign you’re ready for the next big product endeavor.

• If you want your new product to meet with success, you must focus on its go-to-market with a dedicated organization, rather than on engineering alone.

Do you have a similar experience? Did you observe these caveats?

20% to 25%! You must be kidding Mathias. Take Oracle, they have 7% and they’re an R&D machine!

I remember this conversation with a CEO I met some years ago, arguing that for a company of the size he was running, that was the expected Engineering budget as a percentage of revenue. “Well, yes, but you’re not yet Oracle 😉 and certain synergies don’t yet apply”.

In today’s fast-evolving software landscape, the dynamic between CEOs and CT/POs (or, in general, between business & finance leaders (CEO, CFO, CRO, CSO) and tech leaders) is both a driving force for innovation and a battleground for differing priorities. Based on my leader’s experience on the tech side across varying sizes and verticals, I’ve witnessed or experienced firsthand how these two pivotal roles — each with its unique focus and mandate — can sometimes find themselves at odds. The tension isn’t born out of malice; rather, it emerges from the fundamental differences in perspective and objectives.

A Clash of Priorities

At its core, the friction between CEOs and CTOs stems from the divergent ways they view success. CEOs, with their overarching responsibility for the company’s bottom line and market position, often make decisions through the lens of immediate financial performance and investor expectations. In contrast, CTOs are champions of long-term innovation and technological robustness, focused on sustainable growth and the future-readiness of the organization. This divergence can lead to clashes on virtually every strategic front.

Key Areas of Disagreement

Before I go through some of these, I’d like to set the scene: I’m looking at established software businesses (independent companies or business units) from 50+ employees up to a few thousand. I will not consider “founder clash” as often met in early startups / seed phase: there are good articles aplenty about this specific situation, while some of the patterns I describe below would also apply. Also, the CEO is not necessarily a founder.

Budget vs. Product Roadmap

• Fiscal Year Pressures: CEOs are tasked with meeting tight financial targets each fiscal year. Their drive to close the year on budget can sometimes lead to decisions that disrupt the established product roadmap. This might mean compromising on the quality or delaying investments in critical innovation just to meet short-term financial goals (which may come back biting the CEO later down the road, but as they then say: “one problem at a time, we’ll see about it when and if it occurs”). For further reading on this particular subject, see the recent article from Chris Barrett, For a Few Dollars More: How to Sell your Product Roadmap.

• Product Integrity: On the other hand, CT/POs are deeply invested in maintaining a coherent and progressive product / technology strategy. For them, a stable, well-executed roadmap is vital to ensure the product remains competitive, serviceable, and scalable over time.

Short-Term Financial Focus vs. Long-Term Innovation

• Immediate Revenue vs. Future Growth: CEOs often need to show tangible, short-term results to satisfy investors and maintain market confidence. This focus on immediate revenue and market performance can sometimes lead to decisions that sideline long-term functional or technological investments.

• The Innovation Investment Dilemma: CT/POs understand that innovation is a marathon, not a sprint. They invest in long-term advancements of product quality attributes (functionality, ease of use, scalability, security – see ISO 25010 for complete reference), even when the direct financial benefits are not immediately apparent. Linking quality improvements and new features directly to increased revenue can be a challenging proposition, creating a persistent tension between immediate financial needs and long-term strategic.

Different Success Metrics and Misaligned Incentives

• Outcome vs. Output: CEOs typically measure success by financial outcomes, market share, customer satisfaction and growth metrics. For them, delivering strong quarterly figures is paramount. Conversely, CT/POs often gauge success by the quality of the technology, the robustness of the infrastructure, and the consistency and number of innovative features developed—even if these don’t immediately translate into higher sales.

• Incentive Misalignment: The problem deepens when the reward structures for CEOs and CT/POs are not aligned. While CEOs might receive bonuses based on short-term financial metrics, CT/POs might be rewarded for product output, sometimes for a significant part. This misalignment can foster a mindset where each leader champions their own metrics at the expense of a unified strategic vision.

Risk Tolerance Discrepancies

• Conservative vs. Calculated Risks: In many cases, CEOs are more risk-averse because their mandate is to protect the company’s fiscal health. This often leads to cautious decision-making aimed at preserving current revenue streams, especially when return over investment is longer than a year.

• Balancing Act: Conversely, CT/POs are sometimes willing to take calculated risks — launching a new product, investing in emerging technologies or re-architecting systems — to drive long-term growth. The irony is that the very innovation they champion might appear too risky from a “fiscal year” perspective. Furthermore, in certain scenarios, a CEO might want to aggressively pursue a new market opportunity, while the CT/PO could advocate for a more measured approach due to concerns over technical debt and product quality.

Communication Gaps

• Different Languages: The divide is exacerbated by the distinct terminologies and frameworks each leader uses. CEOs, though well-versed in the software business, often lack hands-on technical experience and are not familiar with key software architecture concepts (while these may have deep cost implications!). Even a hard and genuine effort to understand these issues can lead to micromanagement very fast. CT/POs, despite their business acumen, may struggle with the intricacies of financial planning and investment case analysis (think discounted cash flows and other financial metrics) – they don’t need to hold an MBA.

• Bridging the Divide: This communication gap can lead to misunderstandings about project statuses and priorities. While a CEO might be focused on high-level business metrics, the CTO might be immersed in technical details, leading to conversations that don’t always align.

Resource Allocation Conflicts

• The “More is Better” Mindset: With limited resources at their disposal, both roles often find themselves at loggerheads over where to invest. CEOs may continuously push for additional resources to fuel growth, sometimes overlooking the need to optimize or even postpone existing priorities.

• Choosing is Renouncing: For CT/POs, every new initiative means a trade-off. The struggle to decide whether to invest in new technology or enhance existing systems often results in conflicts that can derail a unified strategic approach.

Operational Pragmatism vs. Visionary Thinking

• The Tactical and the Strategic: CEOs are naturally drawn to solving immediate operational challenges. They may zero in on customer satisfaction metrics or short-term market performance to steer the company through turbulent times.

• The Future Beckons: CT/POs, however, are often the visionaries, pushing for future-proofing strategies and experimentation — even if that means chasing the latest shiny technology or product use-case. Conversely, a founder CEO, known for their bold, visionary ideas, clashes with an outcome- and detail-oriented CT/PO who is more focused on running the show for current and onboarding customers, execution, and feasibility.

Different Strategic Time Horizons

• Mid-Term vs. Long-Term: For many established software companies, CEOs work with a mid-term plan spanning three to five years. In contrast, CT/POs often view technology as a long-term investment, especially when scaling operations globally or building a tech platform designed to last a decade or more.

• Temporal Tension: This difference in planning horizons can lead to strategic misalignment, where immediate business pressures overshadow investments that could secure future competitive advantage.

Opposing Leadership Styles

• Decisive vs. Collaborative: The personal dynamics between a decisive, market-driven CEO and a collaborative, execution-oriented CT/PO can create friction. The CEO’s propensity to make swift decisions in a fast-paced market can be at odds with the CT/PO’s methodical, consensus-driven approach.

• Personality Clashes: Moreover, differing levels of ambition and assertiveness can intensify the divide. When each leader is deeply invested in their perspective, the gap can widen, making it challenging to reach a common ground on strategic directions.

Actionable Takeaways for Bridging the Divide

Luckily and as always, there are ways to improve! Here’s a few I could think of, and I’m looking forward to additional insights from the community.

1. Establish Common Goals: Create a unified vision that integrates both short-term financial targets and long-term innovation goals. This involves crafting shared KPIs that value both immediate outcomes and the underlying technical achievements that drive future success. With that respect, Customer Satisfaction is certainly a leading indicator along which they can align their time reference and objectives. A product P&L and investment business cases are also powerful tools for positive and fact-based joint conversations.

2. Foster Regular Dialogue: Encourage ongoing and structured communication between CEOs and CT/POs. Regular strategy sessions, joint reviews, and cross-departmental workshops can help both parties understand each other’s perspectives and priorities. Note that I’m not referring to the regular 1-1’s or management team meetings but dedicated working sessions.

3. Align Incentives: Revise reward systems to reflect a balanced approach. Consider bonus structures or performance metrics that reward collaboration and shared successes rather than siloed achievements.

4. Bridge the Knowledge Gap: Invest in cross-training initiatives where CEOs gain a deeper understanding of technological, domain or process imperatives, and CT/POs enhance their financial literacy. This mutual education can demystify each other’s domains and reduce miscommunication.

5. Implement Conflict-Resolution Mechanisms: Develop a clear process for resolving disagreements that leverages the strengths of both roles. This might include mediators from the broader leadership team or external advisory support to ensure that conflicts are addressed constructively.

6. Adopt a Flexible Strategic Framework: Utilize agile planning methodologies that allow the company to pivot quickly in response to market changes, while still investing in long-term projects. Establishing agreed “rules of engagement” can also help: consider for example bandwidth guardrails (capacity reservation) for quality, research & prototyping, technical debt, etc., together with a simple way to agree why to change (triggers). This balance can help manage the inherent tension between immediate operational needs and future innovation.

7. Promote a Culture of Mutual Respect: At the heart of bridging the CEO-CT/POs divide is respect for the unique contributions each leader brings to the table. Cultivating an environment where both perspectives are valued can transform potential conflicts into opportunities for constructive collaboration. In other words, as always, trust is key.

Conclusion

The divide between CEOs (or business and financial leaders at large) and CT/POs is not merely a clash of personalities but a reflection of the broader challenge of balancing immediate business imperatives with the long-term needs of technology innovation. As the software industry continues to evolve, companies that successfully bridge this gap will be better positioned to adapt to market changes, drive sustainable growth, and innovate continuously.

For tech businesses, recognizing and addressing these tensions is crucial. By fostering open communication, aligning incentives, and creating a strategic framework that embraces both immediate and long-term goals, organizations can transform a potential source of conflict into a competitive advantage.

Lastly, let me insist that such (short-long) tensions are necessary and … inevitable, and every single business will go through these. Pr. Greiner addressed this brilliantly in his HBR paper Evolution and Revolution as Organizations Grow first published in … 1972!, and I cannot encourage you enough to read it and reflect in which phase your company is along Pr. Greiner’s growth curve: it could be an eye-opener.

What about you?

Have you experienced the challenges of navigating the CEO-CT/POs divide in your organization? Did I miss a key opposition area?