--- title: "Rollback" description: "Rollback guidance for Guardian Platform upgrades." source: "Guardian Platform" --- Rollback restores the platform to a previous approved release when an upgrade cannot be accepted. Plan rollback before upgrading. ## Before Rolling Back Confirm whether the failed upgrade changed platform data. Services apply their database migrations when the new release starts, and a Helm rollback does not reverse them. If the previous release cannot run against the migrated schema, or the release notes identify non-reversible changes, restore the databases from the pre-upgrade backup and follow the release-specific recovery plan. ## Rolling Back from 1.3 to 1.2.x Returning a 1.3 installation to 1.2.x requires restoring the pre-upgrade backup. A `helm rollback` alone leaves the platform partly unusable, because three 1.2.x services can't work against the schema 1.3 migrated: | Service | 1.2.x against the 1.3 schema | Needs the restore on its own | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | | Auth Service | Runs | No | | Governance Service | Runs | No | | LLM Gateway | Starts, but refuses every request that names a project with `project_resolver_error`, because 1.3 drops the table of project references it reads | Yes | | Integrity Service | Doesn't start. It refuses the migrations it doesn't know: `migration ... was previously applied but is missing in the resolved migrations` | Yes | | Control Plane | Starts, but its agent and project routes fail, because 1.3 drops the table of project references they read. With browser sign-in on, sign-in also fails, because 1.3 drops the tables that hold its identities and sessions | Yes | Because these services need it, restore the whole platform from the one pre-upgrade backup, following the [Rollback Flow](#rollback-flow): the PostgreSQL databases, OpenBao storage if Auth Service signs with OpenBao, and object storage from the same point in time. Anything recorded after the upgrade is lost. - **Roll back the chart and its values together.** A 1.2.x Control Plane turns on browser sign-in by default, which 1.3 no longer configures, so it doesn't start with 1.3 values. `helm rollback` restores both. - **Protect the pre-upgrade backup.** 1.3 replaces every header value outside an allowlist of standard protocol and provider headers with `[REDACTED]` in stored LLM Gateway audit records, including signatures, API keys, content digests, and `X-Guardian-Correlation-Id`. No down migration brings them back. The backup still holds those values, so restrict access to it as you would to the credentials themselves, and delete it under your retention policy once the upgrade is accepted. - **OpenBao installations return to a 1.2.1 limitation.** After the restore, a 1.2.1 Control Plane and LLM Gateway reject project membership credentials signed with an OpenBao (P-256) key. See [Known Issues](./known-issues.mdx). - **The same limits apply during the upgrade.** Once the LLM Gateway has migrated, a 1.2.x Control Plane fails its agent and project routes and a 1.2.x LLM Gateway pod refuses requests that name a project, until the last 1.2.x pod is replaced, which is why the gateway stack upgrades in two steps. See [LLM Gateway Operations](../llm-gateway/operations.mdx#before-upgrades). ## Rollback Flow 1. Stop or pause affected user workflows if required by the incident plan. 2. If the release-specific plan requires a restore, restore before rolling back the chart, so no service runs against data that is about to be replaced: 1. Scale the platform's service Deployments to zero, leaving the bundled PostgreSQL running if you use it. 2. Restore secrets, then the databases, OpenBao storage, and object storage from the same backup, as [Restore Order](./backup-and-restore.mdx#restore-order) steps 2–4 describe. 3. Restore the previous chart version and values, either with `helm rollback` to the last good revision or by re-running the install command with the previous `--version` and values files: ```bash helm history guardian-platform --namespace guardian helm rollback guardian-platform --namespace guardian ``` A release installed before 1.3 is `governance-platform` in the `governance` namespace; use those names instead. Rolling back the chart also restores the previous replica counts, which brings the services back on the restored data. 4. Verify sign-in, project access, attachments, reports, indicators, lineage, and certificates. 5. Record the incident, release versions, evidence, and customer decision. ## Support Notes Keep the release manifest, previous values, current values, and validation evidence with the support record. Do not continue retrying an upgrade until the rollback cause is understood. ## Related - [Upgrade](./upgrade.mdx) — the upgrade process and the rollback artifacts to keep - [Backup and Restore](./backup-and-restore.mdx#restore-order) — the order to restore data, secrets, and charts - [Release Manifest](./release-manifest.mdx) — the previous chart version and images to restore - [Deployment Troubleshooting](./troubleshooting.mdx) — first checks before deciding to roll back