Disaster Recovery

Restore Develocity to an earlier state from a database backup, for both the embedded database and a user-managed one.

A restore requires downtime, so schedule it as a maintenance window.

A restore operation overwrites current data. Before proceeding, confirm you have the correct backup archive. Everything Develocity recorded between that backup and now is lost.

Restoring an Embedded Database

If you are using the embedded database, restore it with the Develocity CLI.

Standalone Installations

Identify the Backup: Locate the .zip backup archive in /opt/gradle/backups (or your custom off-host storage location).
Run the Restore: Use the following command, replacing the placeholder with your specific filename:
```
develocityctl backup restore /opt/gradle/backups/backup-«date-time».zip
```

Kubernetes Installations

Identify the Backup: If you have copied backups off-cluster, use one of those. If you need to copy a backup from the cluster, use develocityctl backup copy:
```
develocityctl backup copy ./backup.zip
```
Run the Restore: Use the following command with the path to the backup archive:
```
develocityctl backup restore ./backup.zip
```

Monitoring the Restore

The CLI scales down the Develocity deployments before restoring the data, then scales them back up.

Found Develocity installed in the 'develocity' namespace.
Copying backup
Restarting
deployments.apps/edge-node scaled
deployments.apps/gradle-monitoring scaled
deployments.apps/gradle-enterprise-operator scaled
...
Done. Backup will be restored.

If horizontal scaling is enabled, the restore command resets the replica counts to the values defined in your configuration.

Restoring a User-Managed Database

For a user-managed database such as PostgreSQL or Amazon RDS, restore the database with your provider’s own tools.

Scenario A: In-Place Restore

Use this if you are restoring data to the same database instance.

Stop Develocity: To prevent data corruption, stop the application services.
```
develocityctl system stop
```
Restore Data: Follow your provider’s instructions (e.g., pg_restore or a cloud snapshot restoration).
Verify Permissions: If you do not provide superuser credentials to Develocity, re-run your database setup scripts. They set the permissions for the application role, which serves requests, and the migrator role, which runs schema migrations.
Start Develocity:
```
develocityctl system start
```

Scenario B: Restore to a New Instance

Use this if you are moving data to a different database server.

Stop Develocity: Ensure the old instance is no longer receiving traffic via develocityctl system stop.
Restore to New Host: Follow your provider’s instructions to restore your backup data onto the new instance.
Update Configuration: Configure Develocity with the connection details for the new instance, and the superuser credentials if you provide them. Re-run Helm, or update the ConfigMap and Secret resources.
Verify Permissions: If you do not provide superuser credentials to Develocity, run the database setup script (see your installation manual) against the new database. Then confirm that the application and migrator passwords work, either in the configuration UI or in the Secret resources.
Start Develocity: Use develocityctl system start to initiate the connection.

Troubleshooting Restoration Failures

If the system fails to start after a restore:

Check Logs: Review the application logs for connection strings or permission errors.

kubectl --namespace=«namespace» logs --selector=app.kubernetes.io/component=enterprise-app

Verify Network: If you migrated to a new database instance, verify that the Develocity Pods can reach the new database host and port. You can test connectivity from within the cluster by running a temporary Pod:
```
kubectl --namespace=«namespace» run --rm -it pg-test --image=busybox --restart=Never -- nc -zv «db-host» «db-port»
```
Generate Support Bundle: If the issue persists, generate a diagnostic bundle and share it with Develocity support.
```
develocityctl support-bundle
```