Rolling back a deployment
In DevSecOps, use either of the following methods to roll back a deployment using:
-
Rollback trigger
-
Raw GitOps
Rollback Trigger
Use the rollback listener to trigger a rollback to a last known good version.
Follow the following steps to create a rollback trigger:
- Go to the CD pipeline.
- Add a manual trigger or duplicate the Manual CD Trigger.
- Edit this trigger to set the listener to
cd-rollback-listener
. - Select Save.
- Create triggers according to the region and target environment combinations.
A rollback pipeline takes in the following inputs as environment properties:
Environment properties | Description |
---|---|
rollback-change-request-id |
The change request ID of the concluded deployment that you want to rollback against. This is a mandatory field. |
rollback-limit |
The integer value specifies how many deployments back you can roll back. The default value is 1 , limiting rollback to the last completed deployment. |
region |
Specifies the region of the rollback. |
target-environment |
Specifies the target environment for the rollback. For example, stage or production. |
The following criteria must be met for a rollback to occur, otherwise the pipeline is terminated:
- The
rollback-change-request-id
must be the change request ID of an older concluded deployment ID for the sameregion
andtarget-environment
. - The
rollback-change-request-id
must not be older thann
concluded deployments older than what's specified forrollback-limit
value. The default value is1
.
Overview of the Rollback pipeline
The rollback pipeline contains the following stages:
- prod-rollback-start
- prod-setup
- prod-rollback-change-request
- prod-deployment
- prod-acceptance-tests
- prod-rollback-finish
A rollback pipeline run uses information from the specified rollback change request to create a new change request.
To reflect current compliance, the pipeline reopens issues linked to the previous rollback. A list of these issues is attached to the change request. Original due dates remain unchanged.
A successful rollback pipeline deployment moves the _latest
tag in the inventory to a previous commit.
PIPELINE_NAME
, a tekton environment property determines a CD pipeline deployment or rollback. The default values cd-rollback-pipeline
and cd-pipeline
indicate rollback and deployment, respectively, in
the CD pipeline. Customize branching logic using this property.Customize branching logic using this property.
Rollback using raw GitOps
Use the continuous deployment pipeline to deploy a previous version of the inventory to the target environment.
To roll back the deployment, complete these steps:
-
Select the version of the deployment (identified by
commit-id
) in the inventory to roll back to. -
Revert the repository state to the identified commit.
-
Create a pull request to promote the reverted state.
The following commands show the scenario by using
git
commands:-
To roll back to the latest stable deployment, list commits and tags to find the corresponding commit ID.
# /c/usr/devsecops/compliance-inventory (master) $ git show-ref --tags 09ce370c549a8313993ee143cbc9abc3127584c8 refs/tags/1 09ce370c549a8313993ee143cbc9abc3127584c8 refs/tags/2 09ce370c549a8313993ee143cbc9abc3127584c8 refs/tags/3 0cc6ddd7dafffdc5a955a640e678633c2710b174 refs/tags/4 0cc6ddd7dafffdc5a955a640e678633c2710b174 refs/tags/5 0cc6ddd7dafffdc5a955a640e678633c2710b174 refs/tags/6 83f7a87ee59185eaeac554bd3abeebfd2c1b4ad8 refs/tags/8 0019d75d466bbdaba02ea979f3e094784a10b558 refs/tags/9 1914a125e76aa97c497f4bd2c2f455b58cf079b8 refs/tags/10 d9dd5d8553889ef24dff0678a3fcbae8aed3259b refs/tags/11 1914a125e76aa97c497f4bd2c2f455b58cf079b8 refs/tags/prod_latest
-
Select the inventory state before reverting to
refs/tags/8
. The following command lists all the versions or commits between the current state (refs/tags/prod_latest
) and the last known good state (refs/tags/8
).# /c/usr/devsecops/compliance-inventory (master) $ git rev-list --no-merges HEAD...83f7a87ee59185eaeac554bd3abeebfd2c1b4ad8 67cc8babdff3e09c1f0e632f897798c1b5424f38 6fab5ce3d60590cd858206424ecfd7d3a8c9ceb4 22a575d48008299116ea426bdac45417d9df6238 cb6f4d53c17f0c2554c039708989c403eb0ead18
-
Revert the inventory state to
refs/tags/8
.# /c/usr/devsecops/compliance-inventory (master) $ git revert -n $(git rev-list --no-merges HEAD...83f7a87ee59185eaeac554bd3abeebfd2c1b4ad8)
-
Commit the new state of the inventory.
# /c/usr/devsecops/compliance-inventory (master|REVERTING) $ git commit -m "revert master to 83f7a87ee59185eaeac554bd3abeebfd2c1b4ad8" [master af82538] revert master to 83f7a87ee59185eaeac554bd3abeebfd2c1b4ad8 2 files changed, 19 insertions(+), 19 deletions(-) rewrite compliance-app (94%)
-
Push the update to the master branch.
# /c/usr/devsecops/compliance-inventory (master) $ git push --set-upstream origin master Enumerating objects: 7, done. Counting objects: 100% (7/7), done. Delta compression using up to 8 threads Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 1.50 KiB | 766.00 KiB/s, done. Total 4 (delta 2), reused 0 (delta 0) To https://us-south.git.cloud.ibm.com/jaunin.b/compliance-inventory.git 67cc8ba..af82538 master -> master Branch 'master' set up to track remote branch 'master' from 'origin'.
-
-
Create a pull request for the rollback promotion pull request.
-
Review the pull request and merge it.
-
Trigger the Manual CD pipeline run in the CD Toolchain.
Here's a summary of all steps that the CD Pipeline follows for forced redeployment:
- The CD Pipeline starts and tags the current commit to the pipeline run ID.
- The pipeline picks up the content of the corresponding environment branch from the specified tag.
- The pipeline calculates the deployment delta between the current commit and the contents of the
<target-environment>_latest
tag. - The
<target-environment>_latest
tag is added to the current commit upon successful deployment.