Gerald Ajam

Library / Glossary / The craft glossary

DevOps and continuous delivery glossary

48 terms from R12, DevOps and continuous delivery — 23 defined in the guide itself and 25 more from the field around it. Every term the guide teaches links to the slide that teaches it.

The whole craft glossary

R12 · How to build

DevOps and continuous delivery

How do we release often without breaking a school day?

Every term below is defined in the words of devops and continuous delivery, guide R12 of craft guides for educational technologists, and opens the guide at the slide where it is taught. 25 of the 48 are the field’s vocabulary rather than the guide’s own: words a reader will meet around this subject, defined here because the guide assumes them. 2 terms are also defined by another guide in the series; where the two differ, both wordings are given. The whole craft glossary holds all of them together.

TermDefinitionReferred to inRead further
B
Blue-green deployment

Running two identical production environments and switching traffic between them, so a release can be reversed quickly (Fowler, 2010).

  • Fowler, M. (2010). Blue green deployment. martinfowler.com. martinfowler.com
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Build artefact

The packaged, versioned output of a build. The pipeline creates it once and deploys that same package to every environment, so what reaches users is exactly what was tested.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
C
Canary release

Rolling a change out to a small group of users first, and widening it only if the signals hold (Sato, 2014).

  • Sato, D. (2014). Canary release. martinfowler.com. martinfowler.com
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Change advisory board

A group that reviews and approves changes before they go live. Often called a CAB.

  • Forsgren, N., Smith, D., Humble, J., & Frazelle, J. (2019). Accelerate: State of DevOps 2019. DORA and Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
Change fail rate

The share of deployments that need immediate intervention afterwards (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
Change freeze

A period in which no changes may be deployed, except agreed urgent fixes.

  • Majors, C. (2019). Friday deploy freezes are exactly like murdering puppies. charity.wtf. charity.wtf
  • Majors, C. (2025). On Friday deploys: Sometimes that puppy needs murdering. charity.wtf. charity.wtf
Change lead time

The time from a change being committed to version control to it running in production (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
Chaos engineering

Deliberately injecting failures, such as shutting down a server, into a system under controlled conditions to check that it copes as designed before a real failure tests it.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Basiri, A., Behnam, N., de Rooij, R., Hochstein, L., Kosewski, L., Reynolds, J., & Rosenthal, C. (2016). Chaos engineering. IEEE Software, 33(3), 35–41. doi
Configuration drift

The gradual divergence of servers or environments that are meant to be identical, caused by manual, unrecorded changes. It makes releases behave differently from one environment to the next.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Morris, K. (2016). Infrastructure as code: Managing servers in the cloud. O'Reilly Media.
Configuration management

Keeping everything needed to build and run a system, including code, settings, scripts and environment definitions, identified, versioned and reproducible, so that any release can be recreated exactly.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Container orchestration

Software that starts, places, scales, restarts and updates containers across a group of servers automatically, to match a declared desired state. Kubernetes is the most widely used.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Burns, B., Grant, B., Oppenheimer, D., Brewer, E., & Wilkes, J. (2016). Borg, Omega, and Kubernetes. Communications of the ACM, 59(5), 50–57. doi
Continuous delivery

Building software so that it can be released to production at any time (Fowler, 2013).

  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
  • Fowler, M. (2013). Continuous delivery. martinfowler.com. martinfowler.com
  • Humble, J. (n.d.). What is continuous delivery? Continuous Delivery. Retrieved September 17, 2026, from continuousdelivery.com
Continuous deployment

Sending every change that passes the pipeline to production automatically, with no human release decision.

  • Fowler, M. (2013). Continuous delivery. martinfowler.com. martinfowler.com
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Continuous integration

Merging every developer's work into the main line at least daily, with each merge built and tested automatically (Fowler, 2024).

  • Fowler, M. (2024). Continuous integration. martinfowler.com. (Original work published 2000) martinfowler.com
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
D
Dark launch

Running new code in production for real users without them being able to tell (Fowler, 2020).

Deploy

To put a new version of software into production. It need not be visible to users.

  • Government Digital Service. (2024). Deploying software regularly. GOV.UK Service Manual. gov.uk
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Deployment frequency

A count of deployments over a period; one of the three DORA measures of throughput (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
Deployment pipeline

The automated stages a change passes through from commit to production (Humble & Farley, 2010).

  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
  • Government Digital Service. (2024). Deploying software regularly. GOV.UK Service Manual. gov.uk
Deployment rework rate

The share of deployments that are unplanned and made because of an incident in production (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
DevOps

A movement of developers and operations staff working together, with automation, feature flags, shared measures and a culture that avoids blame, to make releasing so ordinary that it stops being dangerous (Allspaw & Hammond, 2009).

  • Allspaw, J., & Hammond, P. (2009). 10+ deploys per day: Dev and ops cooperation at Flickr [Conference presentation]. O'Reilly Velocity Conference 2009, San Jose, CA, United States. slideshare.net
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • devopsdays. (n.d.). About devopsdays. Retrieved September 17, 2026, from devopsdays.org
DORA measures

Five measures of software delivery performance, three of throughput and two of instability, read together and never as targets (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
E
Error budget

The amount of unreliability a service may have in a period before releases pause (Beyer et al., 2016).

R11 The amount of failure a reliability target allows over a period; one minus the target.

  • Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Site reliability engineering: How Google runs production systems. O'Reilly Media. sre.google
F
Failed deployment recovery time

How long it takes to recover from a deployment that fails and needs immediate intervention (DORA, 2026).

R22 The time it takes to recover from a deployment that fails and requires immediate intervention (DORA, 2026).

  • DORA. (2026). DORA's software delivery performance metrics. Google Cloud. dora.dev
  • Harvey, N. (2026). A history of DORA's software delivery metrics. DORA. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution Press.
Feature flag

A setting that switches a feature on or off, or for some users only, without changing code (Hodgson, 2017).

  • Hodgson, P. (2017). Feature toggles (aka feature flags). martinfowler.com. martinfowler.com
  • Allspaw, J., & Hammond, P. (2009). 10+ deploys per day: Dev and ops cooperation at Flickr [Conference presentation]. O'Reilly Velocity Conference 2009, San Jose, CA, United States. slideshare.net
Flaky test

An automated test that sometimes passes and sometimes fails on the same code. Flaky tests teach a team to ignore failures, which undermines the pipeline.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Luo, Q., Hariri, F., Eloussi, L., & Marinov, D. (2014). An empirical analysis of flaky tests. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (pp. 643–653). ACM. doi
I
Infrastructure as code

Defining servers, networks and their settings in files kept under version control, so that environments are created and changed by running tested scripts, not by hand.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
  • Morris, K. (2016). Infrastructure as code: Managing servers in the cloud. O'Reilly Media.
M
Maintenance window

A period agreed and announced in advance during which a service may be changed or taken offline, chosen for when the fewest people need it.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Limoncelli, T. A., & Hogan, C. (2001). The practice of system and network administration. Addison-Wesley.
O
Observability (of a system)

How well a team can work out what is happening inside a running system from the logs, metrics and traces it emits, including for problems nobody predicted.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Majors, C., Fong-Jones, L., & Miranda, G. (2022). Observability engineering: Achieving production excellence. O'Reilly Media.
On-call

A rota under which a named engineer is reachable for a set period and must respond within an agreed time when the live system raises an alert.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Site reliability engineering: How Google runs production systems. O'Reilly Media. sre.google
P
Platform team

A team that provides shared tools and services, such as the deployment pipeline, hosting and monitoring, as an internal product, so that product teams can release without building these themselves.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Skelton, M., & Pais, M. (2019). Team topologies: Organizing business and technology teams for fast flow. IT Revolution Press.
Production environment

The live system that real users rely on, as distinct from the environments used for development and testing. Often shortened to 'production' or 'prod'.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
R
Recovery point objective (RPO)

The most data, measured as time before a failure, that an organisation accepts losing. An RPO of one hour requires backups or replication at least hourly.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., & Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1). National Institute of Standards and Technology. doi
Recovery time objective (RTO)

The longest a service may stay unavailable after a major failure before the harm becomes unacceptable. It sets how fast recovery arrangements must work.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., & Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1). National Institute of Standards and Technology. doi
Release

To make a change visible and available to users.

Release candidate

A specific build judged potentially fit for release, which goes to users unchanged unless the remaining stages of testing find a reason to reject it.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Rollback

Returning a service to its previous version after a faulty release.

  • Fowler, M. (2010). Blue green deployment. martinfowler.com. martinfowler.com
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Runbook

Written step-by-step instructions for a routine operational task or for responding to a particular alert, so that whoever is on call can act without working it out afresh.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Site reliability engineering: How Google runs production systems. O'Reilly Media. sre.google
  • Limoncelli, T. A., Chalup, S. R., & Hogan, C. J. (2014). The practice of cloud system administration: Designing and operating large distributed systems (Vol. 2). Addison-Wesley.
S
Site reliability engineering (SRE)

Google's approach to operations, which treats running a service as a software engineering problem: reliability targets are explicit, releases are governed by error budgets and repetitive work is automated away.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Site reliability engineering: How Google runs production systems. O'Reilly Media. sre.google
Small batches

Work broken into pieces that take hours to a couple of days, which shortens the time to feedback and makes problems easier to find and fix (DORA, 2025).

  • DORA. (2025). Capabilities: Working in small batches. Google Cloud. dora.dev
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
Smoke test

A short set of checks run straight after a deployment to confirm the system starts and its most important functions respond, before anything more thorough is tried.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Staged rollout

Opening a deployed change by feature flag to staff, then pilot schools, then half of schools, then everyone, and switching the flag off if a signal goes wrong.

Staging environment

A copy of the live system, made as similar to it as practical, where a release is rehearsed and checked before it goes to production.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
T
Test automation

Having software run the tests, on every change and without a person starting them, so that a fault is reported within minutes of being introduced.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
The Three Ways

The principles said to underlie DevOps: fast flow of work from development to operations, fast feedback in the other direction, and a culture of continual experimentation and learning.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Kim, G., Humble, J., Debois, P., & Willis, J. (2016). The DevOps handbook: How to create world-class agility, reliability, and security in technology organizations. IT Revolution Press.
Toil

Operational work that is manual, repetitive, automatable and without lasting value, and that grows as the service grows. Reliability teams cap the share of their time spent on it.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Beyer, B., Jones, C., Petoff, J., & Murphy, N. R. (Eds.). (2016). Site reliability engineering: How Google runs production systems. O'Reilly Media. sre.google
Trunk-based development

A way of working in which developers merge small changes into one shared main branch at least daily, keeping any other branches short-lived.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution. itrevolution.com
  • Fowler, M. (2024). Continuous integration. martinfowler.com. (Original work published 2000) martinfowler.com
V
Virtual machine

A software imitation of a whole computer, with its own operating system, running alongside others on shared physical hardware. Cloud servers are usually virtual machines.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Popek, G. J., & Goldberg, R. P. (1974). Formal requirements for virtualizable third generation architectures. Communications of the ACM, 17(7), 412–421. doi
Z
Zero-downtime release

A release that moves users from the old version to the new one without the service becoming unavailable, so it need not wait for an overnight or weekend slot.

  • R12DevOps and continuous deliveryFrom the field · not in the guide
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley. oreilly.com
Singapore