Data Restoration

More information about restoring Elasticsearch data can be found here.

Considerations

Keep the following in mind when restoring data from a snapshot:

  • You can only restore an existing index if it’s closed and the index in the snapshot has the same number of primary shards

  • You cannot restore an existing open index

  • The restore operation automatically opens restored indices

  1. Get a list of available snapshots ordered by descending start time.

    1. GET _snapshot/continuous_backup/*?order=desc

    1. GET _snapshot/retention_backup/*?order=desc

  2. Get a list of available snapshots from a specific date

    1. GET _snapshot/continuous_backup/fam-backup-2022.08.02-*?verbose=false

    1. GET _snapshot/retention_backup/retention_backup-2022.08.02-*?verbose=false

Restore a Deleted Index

To restore a deleted index or indices, find the specific snapshots which contain the index you want to restore (you can use the above examples to find the relevant snapshot).

Copy 
POST _snapshot/retention_backup/retention-backup-2022.08.01-00:10:00/_restore
{  
 "indices": "events_2022_07_2, events_2022_05_1"
}

Restore an Existing Index

If needing to restore an existing index, there are two preferable ways to do it:

  1. Delete and Restore

    https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-restore-snapshot.html#delete-restore

    In case you only need to restore a specific index, the simplest way to avoid conflicts is to delete an existing index before restoring it.

    Example: DELETE pii-1, pii-8

    In the restore request, explicitly specify the repository name, snapshot name, and any indices to restore.

    Copy 
    POST _snapshot/continuous_backup/fam-backup-2022.08.03-09:00:00-fv59i0lpqjipxdtcwirs8a/_restore
    {  
     "indices": "pii-1", "pii-8"
     }

  1. Rename and Restore

    https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-restore-snapshot.html#rename-on-restore

    If you want to avoid deleting existing data, you can instead rename the indices you restore. You typically use this method to compare existing data to historical data from a snapshot. For example, you can use this method to review documents after an accidental update or deletion.

    Copy 
    POST _snapshot/my_repository/my_snapshot_2099.05.06/_restore
    {  
     "indices": "my-index,logs-my_app-default",  
     "rename_pattern": "(.+)",  
     "rename_replacement": "restored-$1"
     }

    When the restore operation is complete, you can compare the original and restored data. If you no longer need an original index, you can delete it and use a reindex to rename the restored one.

    1. To delete the original index: DELETE my-index

    2. To reindex the restored index and rename it: POST _reindex

      Copy 
      {  
       "source": {    
         "index": "restored-my-index"  
        },  
        "dest": {    
         "index": "my-index"  
        }
      }

       

Restore an Entire Cluster

Caution: This should only be used in case of a failure.

Note: File Access Manager recommends reading the Elasticsearch guide first which can be accessed here.

  1. Temporarily stop indexing and turn off the following features:

    GeoIP database downloader

    Copy 
    PUT _cluster/settings
    {  
     "persistent": {    
       "ingest.geoip.downloader.enabled": false  
     }
    }

    ILM

    Copy 
    POST _ilm/stop

     

    Monitoring

    Copy 
    PUT _cluster/settings
    {  
     "persistent": {    
      "xpack.monitoring.collection.enabled": false  
     }
    }

    Machine Learning

    Copy 
    POST _ml/set_upgrade_mode?enabled=true

    Watcher

    Copy 
    POST _watcher/_stop

  2. Use the cluster update settings API to set action.destructive_requires_name to false. This allows you delete data streams and indices using wildcards.

    Copy 
    PUT _cluster/settings
    {  
     "persistent": {    
      "action.destructive_requires_name": false  
     }
    }

  3. Delete all existing data streams on the cluster.

    Copy 
    DELETE _data_stream/*?expand_wildcards=all

  4. Delete all existing indices on the cluster.

    Copy 
    DELETE *?expand_wildcards=all

  5. Restore the entire snapshot, including the cluster state. By default, restoring the cluster state also restores any feature states in the snapshot.

    Copy 
    POST _snapshot/my_repository/my_snapshot_2099.05.06/_restore
    {  
     "indices": "*",  
     "include_global_state": true
    }

Note: Restore request return immediately. The restore happens in the background and the user needs to wait while it completes.

The GET _cluster/health request can be used to monitor Cluster Health and restore progress. See below for the Health request example of response. Green status indicates that the cluster is fine and the restore is complete.

  1. When the restore operation is complete, resume indexing and restart any features you stopped.

GeoIP database downloader

Copy 
PUT _cluster/settings
{  
 "persistent": {    
  "ingest.geoip.downloader.enabled": true  
 }
}

 

ILM

Copy 
POST _ilm/start

 

Machine Learning

Copy 
POST _ml/set_upgrade_mode?enabled=false

 

Monitoring

Copy 
PUT _cluster/settings
{  
 "persistent": {    
  "xpack.monitoring.collection.enabled": true  
 }
}

 

Watcher

Copy 
POST _watcher/_start

  1. Reset the action.destructive_requires_name cluster setting.

    Copy 
    PUT _cluster/settings
    {  
     "persistent": {    
      "action.destructive_requires_name": null  
     }
    }