Automating change management

Pre-deployment readiness assessment

Before a change request is created, the pipeline calculates Deployment Readiness and sets the DEPLOYMENT_READY flag to true or false. This flag is derived from evidence collected across the CI and CD stages. If any evidence check indicates a deviation, or a missing or an unsuccessful check, scan, or test related to the deployed set of artifacts, DEPLOYMENT_READY is set to false.

The change request is prepared with the following fields sourced from the last merged Promotion PR to the target branch:

• risk
• impact
• priority
• assignee
• description
• purpose
• customer impact
• deployment impact
• backout plan

Change request creation

The prepared change request is submitted in one of two initial states depending on DEPLOYMENT_READY:

The change management system also automatically approves change requests when the implementation does not cause any downtime (outage duration is zero) and DEPLOYMENT_READY is true, and deployment risk is within acceptable limits.

If your changes require planned downtime, you must create the change request manually and send it for approval. After it is approved, you can start deployment by providing the change request ID. The pipeline checks its approval state and then runs the deployment. For more information, see Manually approving change requests.

Pre-deployment attachments

Immediately after the change request is created, the pipeline attaches the following artifacts to the CR record:

• Deployment BOM — lists all components included in the deployment
• Delta Summary — evidence posture of all the components participating in the deployment
• Evidence Checks Config File — attached only if required evidence checks based gating is configured in the pipeline
• SCC Profile — attached only if a Security and Compliance Configuration is configured

Approval gate

If the change request was created as Not Approved, it is placed in an unapproved state and sent for human review. No deployment occurs until approval is granted.

You can read the created change request ID from the pipeline logs, wait for approval, and then restart the deployment by using the same change request ID. The pipeline checks the approval state and continues the deployment.

Deployment and acceptance tests

With the CR in Implement state, the pipeline executes:

• CD Deployment — promotes code to the target environment
• Acceptance Tests — validates the deployment outcome

This two are user-driven run-stages.

Post-deployment CR closure

When deployment and acceptance tests pass, the pipeline attaches closing artifacts and closes the change request:

• Closing Summary — a summary of the evidences for all the inventory entries at the target commit level.
• Merged SBOM — the post-deployment software bill of materials across all the components in the inventory.

The CR is then closed with a close_category based on DEPLOYMENT_READY at closure time:

Types of change

Change Request Management supports two types of change: emergency or regular.

If the current change is an emergency change, add the emergency label to the promotion pull request.

There is no emergency flow on the CI pipeline side. However, setting the CI pipeline/trigger property skip-inventory-update-on-failure to an empty value or 0 allows the inventory repository to be updated even if issues are detected in the CI pipeline run. With this updated inventory, an emergency change can be activated.

Choosing a recovery path

Path 1: Full rollback using the dedicated rollback listener

If a last-known-good configuration exists — that is, a previously deployed state that is confirmed stable — the fastest path to service restoration is a full rollback. This uses a dedicated rollback listener that is purpose-built for this scenario and does not require a new build or promotion.

For step-by-step instructions and the parameters to configure, see Full rollback using the dedicated rollback listener.

Path 2: Fix-forward as an emergency change

If no viable rollback target exists, or if the investigation has already produced a patch, the fix can be deployed as an emergency change. This path short-circuits the standard evidence gating logic: the pipeline allows the change to be implemented immediately, and the change request is subject to retroactive review and approval after the incident is resolved.

Using this path means accepting that the code deployed to production may still contain unresolved vulnerabilities or open evidence gaps. Service restoration is treated as the higher priority, and outstanding compliance items must be addressed after the incident is closed.

These two paths are not mutually exclusive. In practice, teams may initiate a full rollback first to restore service immediately, and then follow up with a fix-forward once the patch is ready and validated. The sequencing is left to the operator's judgement based on the situation at hand.

Fix-forward procedure

To deploy a fix as an emergency change during a CIE, follow these steps:

1. Rebuild the affected component. Run the CI pipeline to build a new artifact version containing the fix. If CI evidence checks are failing due to incident conditions, set the pipeline or trigger property skip-inventory-update-on-failure to an empty value or 0 to allow the inventory to be updated despite the failures, so that the emergency change can proceed.

2. Promote the fix through environments. Create promotion pull requests starting from the lowest environment and promote upward toward production. If time permits, verify the fix at each stage before promoting further. If the situation is critical, promote directly to production and reconcile lower environments after service is restored.

3. Apply the emergency label. On the promotion pull request targeting production, add the emergency label. This signals to the pipeline that standard evidence gating should be bypassed and the change should be treated as an emergency deployment.

4. Deploy to production. Run the CD pipeline. The pipeline detects the emergency label, skips the approval wait, and proceeds immediately to deployment and acceptance tests. The change request is created and closed with close_category = successful with issues, reflecting that the change was deployed under emergency conditions.

5. Reconcile lower environments. After the production incident is resolved, deploy the same emergency fix artifact to lower environments — staging, pre-production, and so on — so that all environments are in a consistent state with production. If lower environments were verified before promoting to production, confirm that the same artifact version is in place across all tiers.

Post-CIE obligations

An emergency deployment incurs compliance obligations that must be addressed after the incident is closed:

• The change request must be retroactively reviewed and approved by the appropriate approvers.
• Any evidence gaps accepted during the emergency — unresolved vulnerabilities, incomplete scans, or failed checks — must be remediated and the pipeline re-run under standard conditions.
• A root cause analysis (RCA) should be conducted and documented.

The change request record, including the Delta Summary, Closing Summary, and Merged SBOM attached by the pipeline, serves as the primary audit trail for the post-CIE review.

Invoking the emergency flow

The emergency flow is user-initiated:

1. The user runs the pipeline with the emergency label applied to the promotion pull request.
2. The pipeline detects the emergency designation and deployment and acceptance tests proceed immediately without waiting for standard approval.
3. The CR is closed with the closing notes as successful with issues.

If the current change is an emergency change, add the emergency label to the promotion pull request before running the pipeline.

Post-emergency deployment

Once the emergency flow completes deployment and tests, it rejoins the standard flow at the post-deployment stage:

• Closing Summary and Merged SBOM are attached to the CR.
• The CR is closed following the same DEPLOYMENT_READY-based close_category logic as the standard flow.

If the CR type is emergency, the change request must be retroactively reviewed and approved after deployment.

Triggering condition

The inline-rollback flow is entered when deployment or acceptance tests do not pass. The pipeline then evaluates whether inline-rollback is enabled:

• Inline-Rollback not enabled: The CR is left open with close_category = unsuccessful and the pipeline exits. No automatic recovery is attempted.
• Rollback enabled: The rollback script is executed to revert the target environment, and rollback artifacts are gathered and attached to the CR.

Inline Rollback execution

When inline-rollback is enabled, the pipeline:

1. Executes the inline-rollback script if deployment or acceptance test has failed in the CD pipeline.
2. Collects the following artifacts:
   • Rollback Logs — output from the rollback script execution
   • Closing Summary — reflects the rollback outcome
   • Merged SBOM — post-rollback software bill of materials
3. Attaches all three artifacts to the open CR record.

CR closure after rollback

After inline-rollback and artifact attachment, the CR is left open with close_category = unsuccessful. This signals to change management that the deployment was attempted, failed, and was automatically reverted.

A CR with close_category = unsuccessful is the expected and correct outcome when a deployment is reversed - not an indication of a process failure. Operations teams should use the attached rollback logs to investigate root cause.

Running pipeline with a pre-approved change request

You can use a pre-approved change request (CR) for deployment. Two scenarios are possible:

When the CD pipeline recognizes the CR as created by an earlier CD pipeline run, it fast-paths the deployment by:

• Reusing the precomputed delta and evidence summary from the CR evidence.
• Skipping peer review and signature verification steps.
• Deploying the pre-computed delta.

When the CD pipeline cannot determine whether the CR was created by an earlier CD pipeline run, or the provided CR does not match the deployment target of the current run:

• It does not reuse any precomputed delta or evidence summaries.
• It does not skip peer-review or artifact signature verification.
• It recomputes delta and summary from scratch.
• It does not create a new CR, since one was already supplied.

Rerunning pipeline against a failed deployment

If you do not want to use automated change management, you can provide a previously created and approved change request instead. Rerun failed deployments in the following scenarios:

• The latest automatically created change request is not ready for deployment and was not auto-approved. You received approval and must restart deployment using the same change request.
• The deployment requires downtime. You created the change request, it was approved, and you followed your organization's Change Management Policy.
• No code or configuration changed. You created the change request, explained what changed, received approval, and started a deployment using the approved change request.

The change request (CR) remains open after the CD pipeline completes.

You can start the DevSecOps reference continuous deployment pipeline by using a pre-approved change request and entering the change request ID for the change-request-id property.

If the change-request-id property is set, the pipeline skips data collection for the change request and moves ahead to check the approval state. If change-request-id is set to notAvailable by default, a change request is automatically created by the pipeline.