Skip to content
Platform Signals
Go back
Practice

Feature Flags and Progressive Delivery: Shrinking the Blast Radius

11 min read

Deployment and release used to be the same event. Feature flags broke that assumption apart, and in doing so, changed what it means to ship safely.

For most of software engineering history, “deploying code” and “releasing a feature” were the same moment. You merged, you built, you deployed, and the instant that deployment completed, every user on the planet got the new behavior, all at once, with no way to partially undo it short of rolling back the entire deployment. This coupling is the source of an enormous fraction of historical outages: a single bad code path, bundled with nine good ones in the same deploy, taking down the whole release.

Feature flags exist to break that coupling. A feature flag turns “is this code active” from a build-time property into a runtime property — a decision that can be changed in seconds, for a specific subset of users, without a new deployment, and reversed instantly if something goes wrong. This single architectural shift is the foundation underneath the broader practice of progressive delivery: rolling out change gradually, to an increasing audience, while watching real signals, instead of releasing to everyone simultaneously and hoping.

Decoupling Deploy From Release

The core mental model shift is worth stating plainly: deploying code means the new code exists somewhere in production. Releasing a feature means real users are actually experiencing it. Feature flags let these happen at completely different times, for completely different reasons.

This decoupling enables a set of practices that are awkward or impossible when deploy and release are the same event:

The Progressive Delivery Ladder

Progressive delivery is the practice of using flags (or, at the infrastructure layer, similar mechanisms like traffic-splitting) to expose a change to an increasing percentage or increasingly important segment of traffic, with explicit checkpoints to verify health before proceeding to the next stage.

A typical rollout ladder looks like:

  1. Internal only. The flag is enabled only for employees or an internal test account, verifying the feature works at all in production, against production data and production-scale infrastructure, before any real customer sees it.
  2. Canary percentage. A small percentage of real traffic — often 1-5% — gets the new behavior, chosen either randomly or via a consistent hashing scheme so the same users consistently land in the same group across requests, avoiding a confusing experience where a feature flickers on and off for the same person.
  3. Expanding percentage rollout. Assuming the canary stage shows healthy metrics, the percentage increases in stages — 5%, 25%, 50%, 100% — with a pause at each stage long enough to gather statistically meaningful signal, particularly for effects that only manifest after sustained exposure rather than immediately.
  4. Full release. The flag reaches 100% and, after enough time has passed to be confident the feature is stable, the flag itself gets cleaned up from the codebase — flags that stay in the code indefinitely accumulate as technical debt and conditional-logic complexity that nobody remembers the reason for.

The critical discipline at every stage is defining, in advance, what “healthy” means and what triggers an automatic or manual rollback — the same error rate, latency, and business metric thresholds you’d use for any deployment health check, evaluated specifically against the flagged cohort compared to the control cohort, not just against the system in aggregate.

Segmentation: Targeting Beyond Random Percentages

Random percentage rollout is the simplest targeting strategy, but mature feature flag usage often targets specific segments deliberately, for reasons beyond just risk reduction:

This kind of segmentation requires the flagging system to evaluate targeting rules based on real user attributes at request time — user ID, account tier, region, platform — rather than just a stateless random percentage, which is why mature feature flag platforms function as a full-fledged targeting and evaluation service, not just a boolean toggle store.

Consistent Hashing Keeps Cohorts Stable

A subtle but important implementation detail underlying percentage-based rollouts is how a user gets assigned to the “on” or “off” group. Naively re-randomizing on every request would mean the same user flickers between old and new behavior across page loads within the same session, which is a confusing and sometimes broken experience, particularly for features that depend on consistent state across multiple requests. Mature flagging systems instead hash a stable identifier, typically a user or account ID, into a consistent bucket, so the same user reliably lands in the same cohort for as long as the flag exists, only moving between cohorts if the rollout percentage itself changes. Getting this right is one of the reasons building a flagging system from scratch is harder than it initially looks, and one of the stronger arguments for adopting an existing, battle-tested platform rather than reinventing this specific piece of logic.

The Testing Burden Flags Introduce

Feature flags are not free. Every flag in the codebase represents a branch point, and a codebase with many active flags has a combinatorially larger set of possible code paths than one without — the “old” behavior and the “new” behavior both need to keep working correctly for as long as the flag exists, which is a real, ongoing testing and maintenance cost, not a one-time cost paid at flag creation.

This cost compounds specifically when flags interact with each other. A codebase with three independent flags technically has eight possible combinations of on/off states, and most teams do not have the testing discipline or infrastructure to verify all eight combinations behave correctly — they implicitly test only the combinations that happen to occur in production at any given time, which means a combination that becomes newly possible (say, when a second flag’s rollout percentage increases) can expose an interaction bug that was never actually tested, because it never actually occurred before.

The mitigation is discipline around flag lifecycle, not just flag creation:

Feature Flags as an Incident Response Tool, Not Just a Release Tool

Once a flagging system exists in production, it becomes one of the most valuable incident response tools available, independent of its original release-management purpose. A feature suspected of causing a production issue can be disabled in seconds, for the specific affected segment, without waiting for a new deployment to build and roll out — often the fastest possible mitigation available during an active incident, faster even than a well-rehearsed rollback deployment.

This dual purpose is why feature flag systems deserve the same operational rigor as any other production-critical dependency: the flag evaluation service itself needs to be highly available (a flagging system that’s down when you need to flip a flag during an incident has failed at exactly the moment it mattered most), needs to fail safe with sensible defaults if it becomes unreachable, and needs its own monitoring, so that a flag change is itself an observable, auditable event that shows up clearly in the same timeline used to investigate incidents.

Release Flags Versus Experiment Flags: Different Jobs, Different Rules

It’s worth explicitly distinguishing two purposes that feature flags commonly serve, because conflating them leads to real operational mistakes. A release flag exists purely to control rollout risk, and its end state is always 100% on, followed by removal of the flag and the old code path once the rollout is proven stable. An experiment flag, used for A/B testing, is fundamentally different: it’s designed to keep a population split, say 50/50, for an extended period, specifically so that a statistically valid comparison can be drawn between the two experiences, and there may never be a single correct final state to converge toward until the experiment concludes and a decision is made based on the data.

Treating an experiment flag like a release flag, by getting impatient partway through and pushing it to 100% before reaching statistical significance, invalidates the entire point of running the experiment. Conversely, treating a release flag like an experiment flag, by leaving it at a stable 50% indefinitely just in case, accumulates exactly the kind of permanent conditional complexity that a disciplined flag lifecycle is supposed to prevent. Making this distinction explicit at the moment a flag is created, including tagging it clearly as one type or the other in whatever flagging platform you use, avoids a surprising amount of confusion later about why a given flag still exists and what would actually justify removing it.

Getting Started Without Building a Platform From Scratch

Teams new to feature flagging often either underinvest (scattering ad-hoc boolean environment variables through the codebase with no central visibility or audit trail) or overinvest (building an elaborate internal flagging platform before validating that the practice fits the team’s actual release cadence). A pragmatic middle path:

  1. Start with an existing, mature feature flag platform (open-source or commercial) rather than building evaluation and targeting logic from scratch — this is a solved problem, and the edge cases (consistent hashing for stable rollout cohorts, safe defaults on evaluation failure) are easy to get subtly wrong.
  2. Apply flags first to the riskiest, highest-blast-radius changes — new payment logic, changes to authentication, anything touching the primary revenue path — where the value of gradual rollout and instant rollback is highest.
  3. Define your rollout ladder and rollback thresholds as a written, repeatable checklist before the first real rollout, not improvised the first time a canary stage looks slightly concerning.
  4. Build flag cleanup into your team’s definition of done for any project that introduced a temporary rollout flag — a flag left in the codebase after full rollout is unfinished work, not finished work with an extra step skipped.

The end state worth aiming for isn’t “we have feature flags.” It’s a release process where exposing a risky change to more users is always a small, reversible, observable decision — never a single irreversible event that either goes perfectly or becomes an incident.


Share this post:

Comments 12

  • Isabel Torres

    Decoupling deploy from release is the single mental model shift that made trunk-based development actually feel safe for our team instead of terrifying.

  • Liam Sullivan

    Deploy on Friday, release on Monday is exactly what convinced our on-call rotation to stop dreading Friday afternoon merges.

  • Meera Iyer

    The distinction between release flags and experiment flags should be printed on the wall of every growth team's office. We conflated the two for way too long.

  • Jonas Berg

    Consistent hashing for stable rollout cohorts is a detail I never appreciated until our home-grown flag system randomly re-bucketed users mid-session and broke a feature.

  • Tara Singh

    Flag lifecycle discipline is where we struggle most. We have flags at 100% for over a year that nobody's gotten around to removing, exactly the technical debt this warns about.

  • rootcause_ray

    Using flags as an incident response tool was the unlock for our team. Flipping a flag off in seconds during a live incident beat waiting on a rollback deployment every time.

  • Bianca Ferreira

    The combinatorial testing burden section explains a genuinely nasty bug we had — two independently-fine flags interacted badly the one time both happened to be at a higher rollout percentage simultaneously.

  • Marcus O'Brien

    Geographic and account-tier targeting beyond just random percentage rollout is something we're only now starting to explore. This gave us a good vocabulary for the conversation with product.

  • quietkernel

    Good point about the flag evaluation service itself needing to be highly available with safe defaults. We had an outage where the flagging service went down and every flag silently defaulted to off.

  • Sasha Petrov

    The rollout ladder with explicit checkpoints at each percentage stage is exactly what we needed instead of our current 'vibes-based' decision to expand rollout percentage.

  • Emily Chen

    Solid, practical piece. We're adopting an existing flagging platform instead of building evaluation logic ourselves after reading the section on how easy it is to get subtly wrong.

  • Rahul Verma

    Flag cleanup as part of the definition of done for any rollout project is a great practice. Going to propose adding it to our team's PR template.


Leave a comment


Previous Post
The On-Call Rotation Playbook: Building a Sustainable Schedule
Next Post
Capacity Planning for Peak Traffic: Lessons From Black Friday Incidents