投稿
  1. 首页
  2. 随笔

es备份数据

幻觉图片 • 2022-12-16 • 随笔 • 阅读 9

es备份数据,第1张

es备份数据 备份ES数据的步骤：

1.创建共享目录：
何为共享仓库，其实就是集群中各个节点都能感知到并将数据写入到该仓库的文件。一般一个节点部署在一个服务器上，在哪里，怎样创建一个文件让各个节点都能往里面写数据呢？

使用sshfs在ES集群中每个节点的相同位置挂载一个共享目录。

// 在每个节点上安装sshfs
 sudo apt-get install sshfs

// 选定一个节点的一个目录作为共享目录
mkdir /data/backup
chmod -R 777 /data
// 在每个节点的相同位置创建目录，并挂载共享目录
mkdir /mnt/backup
chmod -R 777 /mnt
sshfs [email protected]:/data/backup /mnt/backup -o allow_other

// 测试运行ES的用户是否有对共享目录的写权限
sudo -u elasticsearch touch /mnt/backup/test

其中参数-o allow_other 解决了不同节点往共享仓库中写数据的权限问题。

2.修改elasticsearch.yml文件，添加path.repo配置
在elasticsearch.yml中增加

path.repo: /mnt/backup

es配置
path.repo: ["/mnt/backup"]

配置共享仓库的位置，重启节点

3.创建了共享目录后就可以在这个共享目录下为集群创建共享仓库

// 创建仓库
PUT _snapshot/my_backup 
{
    "type": "fs", 
    "settings": {
        "location": "/mnt/backup",
        "compress": true
    }
}

4.备份索引数据。

// 针对具体的index创建快照备份，其中snapshot_name 是快照的名字。
PUT _snapshot/my_backup/snapshot_name
{
    "indices": "index_1, index_2"
}

// 5.查看备份状态
GET _snapshot/my_backup/snapshot_name/_status

6.在不同集群之间迁移数据

 // 备份创建好之后，在共享目录/root/backup里是这样的：
-rw-r--r-- 1 root root   31 12月 15 22:14 index
drwxr-xr-x 3 root root 4096 12月 15 22:14 indices
-rw-r--r-- 1 root root   83 12月 15 22:14 metadata-snapshot_name
-rw-r--r-- 1 root root  181 12月 15 22:14 snapshot-snapshot_name

7.在迁移目标的集群上重复上面创建仓库的 *** 作,即步骤2，3

// 8.（将源集群的备份内容（/root/backup里的所有文件），复制到迁移目标的集群仓库目录里）
// 类似批量导入，所以只需要在主节点中恢复仓库中的数据即可？

// 9.使用RESTful API进行备份的恢复，如果索引已经存在目标的集群，需要先关闭索引，恢复数据后在开启
POST /index_name/_close
POST _snapshot/my_backup/snapshot_name/_restore
POST /index_name/_open
// 10.查看恢复的状态
GET _snapshot/my_backup/snapshot_name/_status
数据库设计

es_backup表

字段注释id主键idrepository仓库名称createTime创建时间

es_repository表

字段注释id主键idrepository仓库名称snapshot快照名称indices索引createTime时间result结果
es备份api模板 
public class SnapshotTest {

    private final RestHighLevelClient restClient;


    public SnapshotTest(RestHighLevelClient restClient) {
        this.restClient = restClient;
    }

    
    public void createRepository(){

        String locationKey = FsRepository.LOCATION_SETTING.getKey();
        //共享文件位置
        String locationValue = "/mnt/backup";
        String compressKey = FsRepository.COMPRESS_SETTING.getKey();
        boolean compressValue = true;

        Settings settings = Settings.builder()
                .put(locationKey, locationValue)
                .put(compressKey, compressValue)
                .build();

        PutRepositoryRequest request = new PutRepositoryRequest();
        request.settings(settings);
        //仓库名称
        request.name("your_backup");
        request.type(FsRepository.TYPE);
        request.timeout(Timevalue.timevalueMinutes(1));
        request.timeout("1m");
        request.verify(true);
        AcknowledgedResponse response;
        try {
           response = restClient.snapshot().createRepository(request, RequestOptions.DEFAULT);
            boolean acknowledged = response.isAcknowledged();
            System.out.println(acknowledged+"成功");
        } catch (IOException e) {
            System.out.println("创建共享仓库失败");
            e.printStackTrace();
        }
    }

    
    public void createSnapShoot(){
        CreateSnapshotRequest request = new CreateSnapshotRequest();
        //仓库名称
        request.repository("your_backup");
        //快照名称
        request.snapshot("snapshot_20201203_sun");
        //快照索引
        request.indices("cpu-2020.12.02-000001");
        request.indicesOptions(IndicesOptions.fromOptions(false, false, true, true));
        request.partial(false);
        request.includeGlobalState(true);
        request.masterNodeTimeout(Timevalue.timevalueMinutes(1));
        request.masterNodeTimeout("1m");
        request.waitForCompletion(true);
        CreateSnapshotResponse response;
        try {
            response = restClient.snapshot().create(request, RequestOptions.DEFAULT);
            SnapshotInfo snapshotInfo = response.getSnapshotInfo();
            List indices = snapshotInfo.indices();
            RestStatus status = response.status();
            int status1 = status.getStatus();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    
    public void getSnapShoot(){
        GetSnapshotsRequest request = new GetSnapshotsRequest();
        //仓库名称
        request.repository("your_backup");
        String[] snapshots = { "snapshot_20201203_sun" };
        request.snapshots(snapshots);
        request.masterNodeTimeout(Timevalue.timevalueMinutes(1));
        request.masterNodeTimeout("1m");
        request.verbose(true);
        request.ignoreUnavailable(false);
        GetSnapshotsResponse response;
        try {
            response = restClient.snapshot().get(request, RequestOptions.DEFAULT);
            List snapshotsInfos = response.getSnapshots();

            SnapshotInfo snapshotInfo = snapshotsInfos.get(0);
            //快照的REST状态
            RestStatus restStatus = snapshotInfo.status();
            //快照编号
            SnapshotId snapshotId = snapshotInfo.snapshotId();
            //快照的当前状态
            SnapshotState snapshotState = snapshotInfo.state();
            //有关分片快照过程中发生的故障的信息
            List snapshotShardFailures = snapshotInfo.shardFailures();
            //快照开始时间
            long startTime = snapshotInfo.startTime();
            //快照结束时间
            long endTime = snapshotInfo.endTime();

        } catch (IOException e) {
            System.out.println("获取快照失败");
            e.printStackTrace();
        }
    }

    
    public void restoreSnapShoot(){
        RestoreSnapshotRequest request = new RestoreSnapshotRequest("your_backup", "snapshot_20201203_sun");
        request.indices("cpu-2020.12.02-000001");
        request.renamePattern("cpu-2020.12.02-000001");
        request.renameReplacement("restored_sun");
        request.masterNodeTimeout(Timevalue.timevalueMinutes(1));
        request.masterNodeTimeout("1m");
        request.waitForCompletion(true);
        request.partial(false);
        request.includeGlobalState(false);
        request.includeAliases(false);
        RestoreSnapshotResponse response;
        try {
            response = restClient.snapshot().restore(request, RequestOptions.DEFAULT);
            RestoreInfo restoreInfo = response.getRestoreInfo();
            List indices = restoreInfo.indices();
            System.out.println(indices);
            int status = response.status().getStatus();
            System.out.println(status);
        } catch (Exception e) {
            System.out.println("快照恢复失败");
            e.printStackTrace();
        }
    }

    
    public void getAllIndices(){
        List list = new ArrayList<>();
        GetAliasesRequest request = new GetAliasesRequest();
        GetAliasesResponse alias = null;
        try {
            alias = restClient.indices().getAlias(request, RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
        }
        Map map = alias.getAliases();
        map.forEach((k, v) -> {
            //忽略elasticesearch 默认的
            if (!k.startsWith(".")) {
                list.add(k);
            }
        });
        System.out.println("所有索引");
        for (Object o : list) {
            System.out.println(o);
        }
    }

    
    public void closeIndices(){
        CloseIndexRequest request = new CloseIndexRequest("cpu-2020.12.02-000001");
        AcknowledgedResponse close;
        try {
            close = restClient.indices().close(request, RequestOptions.DEFAULT);
            boolean acknowledged = close.isAcknowledged();
            System.out.println(acknowledged);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}
					
										


					

						
欢迎分享，转载请注明来源：内存溢出
原文地址: https://outofmemory.cn/zaji/5653089.html

					

				
								

										
										
										
										
										
									

								

					
						
							
						 赞
						(0)
					
					

						
							 打赏
						
							
								微信扫一扫 微信扫一扫
							
							
								支付宝扫一扫 支付宝扫一扫
							
						
					

				

				

					

						

							
								幻觉图片
								幻觉图片
								一级用户组
							
						

						

							

								
									
										
									
									0
								
								
									
										
									
									0
								
							

							
							

								
									
										
									
								
							

						

					

				

			
			
            

            	            

            
			

								

					
					

         				
							
								
							 上一篇
						
						2022-12-16
					

				

												

					
					

         				
							下一篇 
           						
							
						
						2022-12-16
					

				

							


			
			

				

					

						发表评论
					

										

						
						

							

								登录后才能评论
							

							
						

					

									

								
 评论列表（0条）

				
    
									

				
											

		

	

	





	


	

		

			

			

				
				

					

						Copyright © 2022 内存溢出 版权所有
						
							湘ICP备2022025235号						
						Powered by
						
							outofmemory.cn
						
					
					

				

			

			

				

					
					
						
						
					
					
						
					
				

			

		

	




























    
    

    

        
            保存