Recovery

If you're backing up ScienceOps, naturally it's quite possible that one day you'll want to perform a restoration. Restoring ScienceOps is useful for both recovering from a disaster and for setting up development or test clusters.

To perform a recovery, you'll first need to perform a backup of ScienceOps. See scienceops-backup and the Backup Documentation for more information on how to do this.

In the event that your backup data is encrypted you'll also need the public key you used to encrypt the backups output file.

Recommendations

When doing a recovery, it's recommended that you start with a fresh ScienceOps instance. This can be either an instance that you've had waiting in the wings (as in a failover instance without any data on it), or an instance that you've just spun up.

How To

  1. Move the ScienceOps backup file onto the master node of the new instance. You can use a command like scp, or if the file is available via a remote location, you can SSH onto the server and run curl or wget to fetch the file.
  2. If you haven't already, SSH into the master node.
  3. If neccessary Grab the public key you used for encrypting your backup file.
  4. Run scienceops-restore *name of backup file* or scienceops-restore *name of backup file* to perform the restoration. This will do 2 things: (1) Add the appropriate data to the MariaDB database and (2) Place the model files onto the server.
  5. Run sudo start scienceops (ubuntu) or sudo systemctl start scienceops (centos) to start ScienceOps.
  6. SSH into the worker machines and run sudo start scienceops-worker (ubuntu) sudo systemctl start scienceops-worker (centos).
  7. Visit the admin webapp on the IP/Hostname of the master node and navigate to the /admin/servers URL. Click the workers tab. One at a time, input each worker IP/hostname into the Add Worker form and click Add Server.

That's it! ScienceOps will begin creating endpoints for each model. To view progress, visit the /admin/models page to watch the individual models build.

Note: The first round of builds will take longer as the runtime environment for each model will need to be recreated on the new server. Subsequent builds will be much shorter.

results matching ""

    No results matching ""