{"id":8690,"date":"2021-04-20T23:11:45","date_gmt":"2021-04-20T20:11:45","guid":{"rendered":"https:\/\/kifarunix.com\/?p=8690"},"modified":"2024-03-18T22:55:47","modified_gmt":"2024-03-18T19:55:47","slug":"backup-and-restore-elasticsearch-index-data","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/backup-and-restore-elasticsearch-index-data\/","title":{"rendered":"Backup and Restore Elasticsearch Index Data"},"content":{"rendered":"\n<p>In this blog post, you will learn how to backup and restore Elasticsearch Index data. Well, there are various reasons for taking data backups. One of the main reason being to protect the primary data against any unforeseen damage as a result of system hardware\/software failure. In case for Elasticsearch, you might be wanting to migrate the data to a new Elastic cluster or for any other reason.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Backing up and Restoring Elasticsearch Index Data<\/h3>\n\n\n\n<p>In ELK\/Elastic stack, an Elasticsearch backup is called a <code><strong><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshot-restore.html\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">snapshot<\/a><\/strong><\/code>. A snapshot can be taken for an entire running Elasticsearch cluster (<em>including all its data streams and indices<\/em>), specific <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/data-streams.html\" target=\"_blank\" aria-label=\"data streams (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"rank-math-link\">data streams<\/a> or specific Elasticsearch indices.<\/p>\n\n\n\n<p>In this tutorial, we will be using a single node Elasticsearch cluster.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"register-elasticsearch-snapshot-repository\"><a href=\"#register-elasticsearch-snapshot-repository\" class=\"rank-math-link\">Register a snapshot repository<\/a><\/h3>\n\n\n\n<p>Before you can take snapshot of the Elasticsearch index\/cluster, you must first register a repository. There are different types of Elasticsearch repositories;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a aria-label=\"Shared file system repository\u2028 (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshots-register-repository.html#snapshots-filesystem-repository\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Shared file system repository<\/a><\/li>\n\n\n\n<li><a aria-label=\"Read-only URL repository\u2028 (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshots-register-repository.html#snapshots-read-only-repository\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Read-only URL repository<\/a><\/li>\n\n\n\n<li><a aria-label=\"Respository plugins (S3, GCP, HDFS, Azure) (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshots-register-repository.html#snapshots-repository-verification\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Respository plugins (S3, GCP, HDFS, Azure)<\/a><\/li>\n<\/ul>\n\n\n\n<p>In this setup, we will use shared file system repository.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Register Snapshot Repository<\/h4>\n\n\n\n<p>To register a file system repository, you need to define the file system location on all the master\/data nodes Elasticsearch configuration file. This is the path\/location in which you want to store your backup\/snapshot.<\/p>\n\n\n\n<p>In our setup, we have mounted our backup disk on <code><strong>\/mnt\/es_backup<\/strong><\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>df -hT -P \/mnt\/es_backup\/<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>Filesystem     Type  Size  Used Avail Use% Mounted on\n\/dev\/sdb1      ext4  3.9G   16M  3.7G   1% \/mnt\/es_backup<\/code><\/pre>\n\n\n\n<p>To define the location of the path to the backup location on Elasticsearch configuration file, use the option, <code><strong>path.repo<\/strong><\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>path.repo: [\"\/mnt\/es_backup\"]<\/code><\/pre>\n\n\n\n<p>You can simply echo this line to the configuration file;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>echo 'path.repo: [\"\/mnt\/es_backup\"]' &gt;&gt; \/etc\/elasticsearch\/elasticsearch.yml<\/code><\/pre>\n\n\n\n<p>Set the ownership of the repository path to <code><strong>elasticsearch<\/strong><\/code> user.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>chown -R elasticsearch: \/mnt\/es_backup\/<\/code><\/pre>\n\n\n\n<p>If you have a multinode cluster, set the same configuration on all master and data nodes.<\/p>\n\n\n\n<p>Once that is done, restart elasticsearch.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl restart elasticsearch<\/code><\/pre>\n\n\n\n<p>Once you have defined the backup\/snapshot location, you can now register it by running the command below. Remember in this setup, we are using a file system repository.<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\ncurl -X PUT \"192.168.57.20:9200\/_snapshot\/es_backup?pretty\" -H 'Content-Type: application\/json' -d'\n{\n  \"type\": \"fs\",\n  \"settings\": {\n    \"location\": \"\/mnt\/es_backup\"\n  }\n}\n'\n<\/code><\/pre>\n\n\n\n<p>When you run the command, you should get the output;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"acknowledged\" : true\n}<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Get Information about Snapshot Repository<\/h4>\n\n\n\n<p>To retrieve information about a registered repository, run the command below;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl -X GET \"192.168.57.20:9200\/_snapshot\/es_backup?pretty\"<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n{\n  \"es_backup\" : {\n    \"type\" : \"fs\",\n    \"settings\" : {\n      \"location\" : \"\/mnt\/es_backup\"\n    }\n  }\n}\n<\/code><\/pre>\n\n\n\n<p>To view all repositories;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X GET \"192.168.57.20:9200\/_snapshot\/_all?pretty\"<\/code><\/pre>\n\n\n\n<p>If you want to delete a snapshot repository;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X DELETE \"192.168.57.20:9200\/_snapshot\/es_backup\/?pretty\"<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-elasticsearch-snapshot\"><a href=\"#create-elasticsearch-snapshot\" class=\"rank-math-link\">Create Elasticsearch Snapshot\/Backup<\/a><\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Create Snapshot of Entire Elasticsearch Cluster<\/h4>\n\n\n\n<p>Once you have registered a snapshot repository, you can now create a snapshot as shown below. &#8220;<em>A repository can contain multiple snapshots of the same cluster. Snapshots are identified by unique names within the cluster&#8221;.<\/em><\/p>\n\n\n\n<p>Take for example, to create snapshot called <code><strong>es_backup_202104192200<\/strong><\/code>, you would run such a command;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X PUT \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200?pretty\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"accepted\" : true\n}<\/code><\/pre>\n\n\n\n<p><em>By default, a snapshot backs up all data streams and open indices in the cluster.<\/em><\/p>\n\n\n\n<p>You can also use the <code><strong>wait_for_completion=true<\/strong><\/code> parameter to specify <em>whether or not the request should return immediately after snapshot initialization (default) or wait for snapshot completion<\/em> like;<\/p>\n\n\n\n<pre id=\"block-b9ec6ae9-0c13-4ae3-a43a-eeca089c60b6\" class=\"wp-block-preformatted\">curl -X PUT \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200?wait_for_completion=true\"<\/code><\/pre>\n\n\n\n<p>See sample contents of the backup\/snapshot directory after the command completes running;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>ls -1 \/mnt\/es_backup\/<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>index-0\nindex.latest\nindices\nmeta-33qzhT82QTmvH4GkWn-vhw.dat\nsnap-33qzhT82QTmvH4GkWn-vhw.dat<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Create Snapshot of Specific Elasticsearch Index<\/h4>\n\n\n\n<p>In my current, I have just a few indices for demo only;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl 192.168.57.20:9200\/_cat\/indices?pretty<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>yellow open filebeat-7.10.1-2021.04.16-000001 XWQ7QQ_9Tpar_rPE5dn0Sw 1 1    24  0  146kb  146kb\nyellow open filebeat-7.12.0-2021.04.19-000001 0sQCK1OTRWiosULRHKQMpw 1 1 66423  0 15.5mb 15.5mb\n...<\/code><\/pre>\n\n\n\n<p>So let&#8217;s say i want to backup a specific index, <code><strong>filebeat-7.12.0-2021.04.19-000001<\/strong><\/code>;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\ncurl -X PUT \"192.168.57.20:9200\/_snapshot\/es_backup\/filebeat_202104192200?pretty\" -H 'Content-Type: application\/json' -d'\n{\n  \"indices\": \"filebeat-7.12.0-2021.04.19-000001\",\n  \"ignore_unavailable\": true,\n  \"include_global_state\": false\n}\n'\n<\/code><\/pre>\n\n\n\n<p>While taking a snapshot, you can include other options such as;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code><strong>ignore_unavailable<\/strong><\/code>: takes the option <code>true<\/code> or <code>false<\/code>.\n<ul class=\"wp-block-list\">\n<li>When set to true, it causes the indices\/data streams that do not exist to be ignored while taking snapshot.<\/li>\n\n\n\n<li>if not defined, snapshot will fail if a data stream or index is missing while taking snapshot.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><code>include_global_state<\/code><\/strong>: can be set to <strong><code>true<\/code><\/strong> to <strong><code>false<\/code><\/strong>.\n<ul class=\"wp-block-list\">\n<li>if set to <code><strong>true<\/strong><\/code>, it causes the snapshot to save the current cluster state as part of the snapshot.<\/li>\n\n\n\n<li>if set to <code><strong>false<\/strong><\/code>, it prevents the cluster global state from being stored as part of the snapshot.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><code>partial<\/code><\/strong>:\n<ul class=\"wp-block-list\">\n<li>if set to false (default), the snapshot will fail if one or more indices in the snapshot do not have all primary shards available.<\/li>\n\n\n\n<li>If set to true, snapshot will take place even if one or more indices in the snapshot do not have all primary shards available.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><code>expand_wildcards<\/code><\/strong>:\n<ul class=\"wp-block-list\">\n<li>used to control whether <strong><code>hidden<\/code><\/strong> and <code><strong>closed<\/strong><\/code> indices will be included in the snapshot, and defaults to&nbsp;<code><strong>all<\/strong><\/code>.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><code>metadata<\/code><\/strong>:\n<ul class=\"wp-block-list\">\n<li>add information such as <em>who took the snapshot, why it was taken, or any other data that might be useful<\/em> to the snapshot.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>See example below;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\ncurl -X PUT \"192.168.57.20:9200\/_snapshot\/es_backup\/filebeat_202104192200?pretty\" -H 'Content-Type: application\/json' -d'\n{\n  \"indices\": \"filebeat-7.12.0-2021.04.19-000001\",\n  \"ignore_unavailable\": true,\n  \"include_global_state\": false,\n  \"metadata\": {\n    \"taken_by\": \"kifarunix\",\n    \"taken_because\": \"test backup\"\n  }\n}\n'\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">View Snapshot Information<\/h4>\n\n\n\n<p>To view information about created snapshots within a specific repository, run the example commands below.<\/p>\n\n\n\n<p>For example, to view information about <code><strong>es_backup_202104192200<\/strong><\/code> snapshot;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X GET \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200?pretty\"<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n{\n  \"snapshots\" : [\n    {\n      \"snapshot\" : \"es_backup_202104192200\",\n      \"uuid\" : \"33qzhT82QTmvH4GkWn-vhw\",\n      \"version_id\" : 7100099,\n      \"version\" : \"7.10.0\",\n      \"indices\" : [\n        \".kibana_task_manager_1\",\n        \"filebeat-7.12.0-2021.04.19-000001\",\n        \"filebeat-7.10.1-2021.04.16-000001\",\n        \".kibana-event-log-7.10.0-000001\",\n        \".async-search\",\n        \".apm-agent-configuration\",\n        \"ilm-history-3-000001\",\n        \".kibana_1\",\n        \".apm-custom-link\"\n      ],\n      \"data_streams\" : [ ],\n      \"include_global_state\" : true,\n      \"state\" : \"SUCCESS\",\n      \"start_time\" : \"2021-04-19T19:57:08.912Z\",\n      \"start_time_in_millis\" : 1618862228912,\n      \"end_time\" : \"2021-04-19T19:57:56.691Z\",\n      \"end_time_in_millis\" : 1618862276691,\n      \"duration_in_millis\" : 47779,\n      \"failures\" : [ ],\n      \"shards\" : {\n        \"total\" : 9,\n        \"failed\" : 0,\n        \"successful\" : 9\n      }\n    }\n  ]\n}\n<\/code><\/pre>\n\n\n\n<p>You can see indices and data streams in the backup snapshot.<\/p>\n\n\n\n<p>To view all snapshots within a repository;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X GET \"192.168.57.20:9200\/_snapshot\/es_backup\/_all?pretty\"<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"restore-elasticsearch-snapshot\"><a href=\"#restore-elasticsearch-snapshot\" class=\"rank-math-link\">Restore Elasticsearch Snapshot\/backup<\/a><\/h3>\n\n\n\n<p>Now, let say you accidentally deleted an index, which you already had backup for. Then it is easy to restore the Elasticsearch snapshot.<\/p>\n\n\n\n<p>According to <a aria-label=\"Elasticsearch snapshot restore (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshot-restore.html\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Elasticsearch snapshot restore<\/a>;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>You cannot restore snapshots from later Elasticsearch versions into a cluster running an earlier Elasticsearch version. For example, you cannot restore a snapshot taken in 7.6.0 to a cluster running 7.5.0.<\/em><\/li>\n\n\n\n<li><em>You cannot restore indices into a cluster running a version of Elasticsearch that is more than&nbsp;one major version&nbsp;newer than the version of Elasticsearch used to snapshot the indices. For example, you cannot restore indices from a snapshot taken in 5.0 to a cluster running 7.0<\/em>.<\/li>\n<\/ul>\n\n\n\n<p>The following table summarizes the snapshot compatibility between cluster versions;<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:100%\">\n<figure class=\"wp-block-table\"><table><tbody><tr><td><\/td><td><strong>Cluster version<\/strong><\/td><td><\/td><td><\/td><td><\/td><td><\/td><\/tr><tr><td><strong>Snapshot version<\/strong><\/td><td>2.x<\/td><td>5.x<\/td><td>6.x<\/td><td>7.x<\/td><td>8.x<\/td><\/tr><tr><td><strong>1.x<\/strong>&nbsp;\u2192<\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><\/tr><tr><td><strong>2.x<\/strong>&nbsp;\u2192<\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><\/tr><tr><td><strong>5.x<\/strong>&nbsp;\u2192<\/td><td><img loading=\"lazy\" decoding=\"async\" width=\"20\" height=\"15\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><\/tr><tr><td><strong>6.x<\/strong>&nbsp;\u2192<\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><\/tr><tr><td><strong>7.x<\/strong>&nbsp;\u2192<\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-no.png\" alt=\"No\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><td><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/doc-icons.s3.us-east-2.amazonaws.com\/icon-yes.png\" alt=\"Yes\" width=\"20\" height=\"15\" title=\"\"><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p><\/p>\n<\/div>\n<\/div>\n\n\n\n<p>In above, we learnt how to take a snapshot of the entire cluster as well as for an individual Elasticsearch index.<\/p>\n\n\n\n<p>So, for demo purposes, let us delete the indices on our current Elasticsearch;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X DELETE \"192.168.57.20:9200\/_all?pretty\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"acknowledged\" : true\n}<\/code><\/pre>\n\n\n\n<p>If you try to list available indices, you will find one of the Kibana indices having been created automatically;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl 192.168.57.20:9200\/_cat\/indices?pretty<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>yellow open .kibana -CjWP5YlSdi5eqt1VpLXng 1 1 1 0 5kb 5kb<\/code><\/pre>\n\n\n\n<p>Next, to try and restore our general\/entire cluster snapshot, <code>es_backup_202104192200<\/code>, then you can run the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X POST \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200\/_restore?pretty\"<\/code><\/pre>\n\n\n\n<p>If some of the indices already exist and open, within the cluster, that matches some of the indices available in the snapshot, you can <strong>either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name<\/strong>.<\/p>\n\n\n\n<p>Sample error;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n{\n  \"error\" : {\n    \"root_cause\" : [\n      {\n        \"type\" : \"snapshot_restore_exception\",\n        \"reason\" : \"[es_backup:es_backup_202104192200\/33qzhT82QTmvH4GkWn-vhw] cannot restore index [ilm-history-3-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name\"\n      }\n    ],\n    \"type\" : \"snapshot_restore_exception\",\n    \"reason\" : \"[es_backup:es_backup_202104192200\/33qzhT82QTmvH4GkWn-vhw] cannot restore index [ilm-history-3-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name\"\n  },\n  \"status\" : 500\n}\n<\/code><\/pre>\n\n\n\n<p>So let us close the complaining index<\/p>\n\n\n\n<pre id=\"block-6fe46ac5-58a6-4d3b-b6ed-f6e3dbd23bd8\" class=\"wp-block-preformatted\">curl -X POST \"192.168.57.20:9200\/ilm-history-3-000001\/_close?pretty\"<\/code><\/pre>\n\n\n\n<p>Sample command output.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"acknowledged\" : true,\n  \"shards_acknowledged\" : true,\n  \"indices\" : {\n    \"ilm-history-3-000001\" : {\n      \"closed\" : true\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<p>If you get such an error;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n{\n  \"error\" : {\n    \"root_cause\" : [\n      {\n        \"type\" : \"illegal_state_exception\",\n        \"reason\" : \"index, alias, and data stream names need to be unique, but the following duplicates were found [.kibana (alias of [.kibana_1\/vSrhd_CyTva5oI1ggwnCuQ]) conflicts with index]\"\n      }\n    ],\n    \"type\" : \"illegal_state_exception\",\n    \"reason\" : \"index, alias, and data stream names need to be unique, but the following duplicates were found [.kibana (alias of [.kibana_1\/vSrhd_CyTva5oI1ggwnCuQ]) conflicts with index]\"\n  },\n  \"status\" : 500\n}\n<\/code><\/pre>\n\n\n\n<p>Delete the indices again;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X DELETE \"192.168.57.20:9200\/_all?pretty\"<\/code><\/pre>\n\n\n\n<p>And immediately run the snapshot restore before .kibana index is auto created.<\/p>\n\n\n\n<p>Now when you run a snapshot restore;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X POST \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200\/_restore?pretty\"<\/code><\/pre>\n\n\n\n<p>You should get;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"accepted\" : true\n}<\/code><\/pre>\n\n\n\n<p>Listing the indices again should now be same as before;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl 192.168.57.20:9200\/_cat\/indices?pretty<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nyellow open filebeat-7.10.1-2021.04.16-000001 QImIEVM9SOKvtDnO1WUyNw 1 1    24 0   146kb   146kb\nyellow open filebeat-7.12.0-2021.04.19-000001 -rYD-nUNR9m10x2W21uAAg 1 1 66423 0  15.5mb  15.5mb\ngreen  open .apm-custom-link                  b6b_dTNPQHOLatvVyw6fUg 1 0     0 0    208b    208b\ngreen  open .kibana_task_manager_1            f2Eg4u8yRvSEk47QU-wwbg 1 0     5 3 132.9kb 132.9kb\ngreen  open .apm-agent-configuration          kMWsZ9kBTW6xeYoe3J4sIA 1 0     0 0    208b    208b\ngreen  open .kibana-event-log-7.10.0-000001   -ZTzLi9zTuOnjcsm2wOhAw 1 0     2 0    11kb    11kb\ngreen  open .async-search                     la4iO9BFTd6qSUzrw7JKNw 1 0     2 2 924.5kb 924.5kb\ngreen  open .kibana_1                         gK9b55LTRCiuwm9sFRTuaQ 1 0  1558 7  10.7mb  10.7mb\n<\/code><\/pre>\n\n\n\n<p>And that is how easy it is to backup and restore an Elasticsearch index data.<\/p>\n\n\n\n<p>To delete a snapshot;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X DELETE \"192.168.57.20:9200\/_snapshot\/es_backup\/es_backup_202104192200?pretty\"<\/code><\/pre>\n\n\n\n<p>In our next guide, you can also restore elasticsearch index data to a different elasticsearch cluster. Links is provided below;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/restore-elasticsearch-data-to-another-cluster\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Restore Elasticsearch Data to another Cluster<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reference<\/h3>\n\n\n\n<p><a aria-label=\"Elasticsearch Snapshot and Restore (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/snapshot-restore.html\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Elasticsearch Snapshot and Restore<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Other Tutorials<\/h3>\n\n\n\n<p><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/setup-kibana-elasticsearch-and-fluentd-on-centos-8\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Setup Kibana Elasticsearch and Fluentd on CentOS 8<\/a><\/p>\n\n\n\n<p><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/setup-multi-node-elasticsearch-7-x-cluster-on-fedora-30-fedora-29-centos-7\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Setup Multi-node Elasticsearch 7.x Cluster on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/update-change-kibana-visualization-index-pattern\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Update\/Change Kibana Visualization Index Pattern<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this blog post, you will learn how to backup and restore Elasticsearch Index data. Well, there are various reasons for taking data backups. One<\/p>\n","protected":false},"author":3,"featured_media":8696,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[72,910,121],"tags":[3455,3457,3462,3460,3463,3461,3456,3459,3458],"class_list":["post-8690","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-monitoring","category-elastic-stack","category-howtos","tag-backup-elasticsearch","tag-backup-elasticsearch-index","tag-create-elasticsearch-snapshot","tag-elasticsearch-backup","tag-elasticsearch-repository","tag-repo-path","tag-restore-elasticsearch","tag-restore-elasticsearch-index","tag-take-snapshot-of-elasticsearch-index","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/8690"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=8690"}],"version-history":[{"count":18,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/8690\/revisions"}],"predecessor-version":[{"id":21837,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/8690\/revisions\/21837"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/8696"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=8690"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=8690"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=8690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}