Skip to content

Add rollback (w db) steps to upgrade page#397

Open
Rachael-Graham wants to merge 10 commits into
kagent-dev:mainfrom
Rachael-Graham:rlg-db
Open

Add rollback (w db) steps to upgrade page#397
Rachael-Graham wants to merge 10 commits into
kagent-dev:mainfrom
Rachael-Graham:rlg-db

Conversation

@Rachael-Graham

Copy link
Copy Markdown
Contributor

Comment thread src/app/docs/kagent/operations/upgrade/page.mdx Outdated
Comment thread src/app/docs/kagent/operations/upgrade/page.mdx Outdated
Comment thread src/app/docs/kagent/operations/upgrade/page.mdx Outdated
Comment thread src/app/docs/kagent/operations/upgrade/page.mdx Outdated
Comment thread src/app/docs/kagent/operations/upgrade/page.mdx Outdated
Comment on lines +129 to +146
1. Check out the migration files for the target kagent version.
```bash
git clone https://github.com/kagent-dev/kagent.git
cd kagent && git checkout v$ROLLBACK_VERSION
```

2. Run `golang-migrate` down to the target schema version for each track. You can check the `go/core/pkg/migrations/core/` directory in the target release to find the highest migration version number.
```bash
# Core track
migrate -path go/core/pkg/migrations/core \
-database "$DATABASE_URL" \
goto <target-migration-version>

# Vector track (if vector features are enabled)
migrate -path go/core/pkg/migrations/vector \
-database "$DATABASE_URL" \
goto <target-vector-migration-version>
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the steps laid out here are slightly easier and can be combined with this section. For example, I'm not sure you need to checkout the migration files locally if you don't want.


3. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back).

> **Note**: Database migration version numbers do not correspond directly to kagent release versions. Always check the migration files in the target release to identify the correct target migration version.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify this? I'm not sure why they wouldn't line up. I believe we use the same versioning for the git tags as we do for the release.

@Rachael-Graham Rachael-Graham force-pushed the rlg-db branch 2 times, most recently from 08311e4 to b5c2d34 Compare June 23, 2026 15:38
Comment thread src/app/docs/kagent/operations/upgrade/page.mdx
@Rachael-Graham Rachael-Graham force-pushed the rlg-db branch 2 times, most recently from 04a0df7 to 33f4b1a Compare June 23, 2026 16:07
kubectl -n kagent scale deploy/kagent-controller --replicas=0
```

4. Reset the core track. The `github://` source references the migration files directly from the release tag without a local checkout. Replace `<rollback-migration-version>` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of your target (rollback) release, which you can browse at `https://github.com/kagent-dev/kagent/tree/v<target-version>/go/core/pkg/migrations/core/`. For the database connection string, see [Database configuration](/docs/kagent/operations/operational-considerations#database-configuration).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For something like: https://github.com/kagent-dev/kagent/tree/v<target-version>/go/core/pkg/migrations/core/, would it be better to put the rollback version ("target version") in step 1's export as an env var and reference it here?

So something like:

export CURRENT_VERSION=<current-version>
export ROLLBACK_VERSION=<previous-version>

Then use it in the step's text with: https://github.com/kagent-dev/kagent/tree/v${ROLLBACK_VERSION}/go/core/pkg/migrations/core/

I feel like it is clearer to provide env vars instead of something like <rollback-migration-version> or <target-version> in general, but maybe that's just me. I think it probably matters more in the bash blocks, but just a suggestion.


The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to.

For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to `v0.9.4` (migrations up to `8`), you use the `v0.10.2` source and `goto 8`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we ground this in a real example if we are doing it in the docs

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So let's say you go back from 0.9.7 to 0.9.6 (where 0.9.7 added migration 5 to the core track).

You would have goto 4 in that case.

image


The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to.

For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`.

@Rachael-Graham Rachael-Graham Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this to a real set of versions but just fyi since this is only available in 0.9.x and later, but 0.10.x isnt actually released yet, i cant show a minor-version rollback yet from 0.10.x to 0.9.x - lmk if that isnt alright

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a hint why it's goto 4

Suggested change
For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`.
For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4` because you want to go back to migration sequence 4 (v0.9.3's `000004`).


### Reset the database before rolling back further

If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the migration files.

@iplay88keys iplay88keys Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing, you need to roll back one minor at a time and there might be data loss going more than one minor back as our approach is backwards compatibility of one minor version back.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in a note on 133 and in the original desc on 94

Rachael-Graham and others added 10 commits June 23, 2026 12:23
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Co-authored-by: Art <artberger@users.noreply.github.com>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Co-authored-by: Art <artberger@users.noreply.github.com>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>

The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to.

For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a hint why it's goto 4

Suggested change
For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`.
For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4` because you want to go back to migration sequence 4 (v0.9.3's `000004`).

kubectl -n kagent scale deploy/kagent-controller --replicas=0
```

4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a hint, if this is the same number as the goto previously I'm assuming?

Suggested change
4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable.
4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable, such as `4` from the previous v0.9.3 `goto 4` example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants