- ElasticSearch
- 1、配置环境
- 1、导入依赖
- 2、配置文件
- 3、配置客户端
- 2、Rest-索引库
- 1、创建索引库
- 1、定义常量字符串保存创建索引库 *** 作
- 2、restclient方式创建索引库
- 2、判断索引库是否存在
- 3、删除索引库
- 4、总结
- 3、Rest-Document
- 1、新增单条文档
- 2、查询单条文档
- 3、修改文档
- 4、删除文档
- 5、批量导入文档
- 4、复杂查询
- 1、query下的查询
- 1、match_all 全查询
- 2、multi_match 多字段查询
- 3、term 精准查询
- 4、range 范围查询
- 5、bool 复合查询
- 6、地理坐标查询
- 7、算法函数查询
- 2、对查询结果的 *** 作，与query同级
- 1、排序&分页
- 2、高亮
- 3、数据聚合
- 1、Bucket聚合&排序&范围
- 2、Metric聚合 min、max、avg
阅读本文前请注意，本文仅仅展示springboot整合es中大部分场景api的 *** 作，对其概念并没有过多的阐述，想获得更完整的文档，请查阅官方文档 https://www.elastic.co/cn/elasticsearch/
在介绍使用之前，先对比一下es与mysql关键字术语描述的比较吧
1、配置环境 1、导入依赖<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-data-elasticsearchartifactId>
<version>7.6.2version>
dependency>
elasticsearch:
host: 192.168.137.157:9200
@Configuration
public class RestClientConfig extends AbstractElasticsearchConfiguration {
@Value("${elasticsearch.host}")
private String host;
@Override
@Bean
public RestHighLevelClient elasticsearchClient() {
final ClientConfiguration clientConfiguration = ClientConfiguration
.builder().connectedTo(host).build();
return RestClients.create(clientConfiguration).rest();
}
}
public class HotelConstants {
public static final String MAPPING_TEMPLATE =" {\n" +
" \"properties\": {\n" +
" \"id\":{\n" +
" \"type\":\"keyword\"\n" +
" },\n" +
" \"name\":{\n" +
" \"type\": \"text\",\n" +
" \"analyzer\": \"ik_max_word\",\n" +
" \"copy_to\": \"all\"\n" +
" },\n" +
" \"address\":{\n" +
" \"type\": \"keyword\",\n" +
" \"index\": false\n" +
" },\n" +
" \"price\":{\n" +
" \"type\": \"integer\"\n" +
" },\n" +
" \"score\":{\n" +
" \"type\": \"integer\"\n" +
" },\n" +
" \"brand\":{\n" +
" \"type\": \"keyword\"\n" +
" },\n" +
" \"city\":{\n" +
" \"type\": \"keyword\"\n" +
" },\n" +
" \"starName\":{\n" +
" \"type\": \"keyword\"\n" +
" },\n" +
" \"business\":{\n" +
" \"type\": \"keyword\",\n" +
" \"copy_to\": \"all\"\n" +
" },\n" +
" \"location\":{\n" +
" \"type\":\"geo_point\"\n" +
" },\n" +
" \"pic\":{\n" +
" \"type\": \"keyword\",\n" +
" \"index\": false\n" +
" },\n" +
" \"all\":{\n" +
" \"type\": \"text\",\n" +
" \"analyzer\": \"ik_max_word\" \n" +
" }\n" +
" }\n" +
" }";
/**
* 创建索引
* @throws IOException
*/
@Test
void createIndex() throws IOException {
CreateIndexRequest request = new CreateIndexRequest("hotel"); // 索引库名
request.mapping(MAPPING_TEMPLATE, XContentType.JSON); // 常量静态导入
// 发起请求
CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
}
/**
* 索引是否存在
*/
@Test
void existsIndex() throws IOException {
GetIndexRequest indexRequest = new GetIndexRequest("hotel");
boolean exists = restHighLevelClient.indices().exists(indexRequest, RequestOptions.DEFAULT);
System.out.println(exists?"已存在":"不存在");
}
/**
* 删除索引
*/
@Test
void deleteIndex() throws IOException {
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("hotel");
AcknowledgedResponse delete = restHighLevelClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
System.out.println(delete.toString());
}
可以发现创建删除都是采用rest风格的去定义的java API，创建请求是都需要传入索引名，注意索引库是不支持修改 *** 作的，如果有需要修改，直接删除重新创建即可
3、Rest-Document 1、新增单条文档/**
* 新增单条数据
* @throws IOException
*/
@Test
void testAddOneDocument() throws IOException {
Hotel hotel = iHotelService.getById(45845L); // 数据库中查询的数据
HotelDoc hotelDoc = new HotelDoc(hotel); // 数据实体转换
// 1、准备Request对象
// 索引库中对id的要求必须是字符串
IndexRequest request = new IndexRequest("hotel").id(hotelDoc.getId().toString());
// 2、准备Json文档
// 将实体类转换为JSON格式的数据
request.source(JSONUtil.toJsonStr(hotelDoc), XContentType.JSON);
// 3、发送请求
client.index(request, RequestOptions.DEFAULT);
}
java api 对应DSL图
2、查询单条文档/**
* 查询单条数据
*/
@Test
void testQueryOneDocuemnt() throws IOException {
GetRequest getRequest = new GetRequest("hotel", "45845");
GetResponse documentFields = client.get(getRequest, RequestOptions.DEFAULT);
String jsonString = documentFields.getSourceAsString();
// 将json格式数据转换为实体
System.out.println(JSONUtil.toBean(jsonString, HotelDoc.class));
}
/**
* 修改文档的方式有两种
* 方式一：全量更新，写入一样的id，就会删除旧文档，添加新文档
* 方式二：局部更新，只更新部分字段
*/
@Test
void testUpdateDocumentById() throws IOException {
/* Hotel hotel = iHotelService.getById(45845L);
hotel.setScore(46);
HotelDoc hotelDoc = new HotelDoc(hotel);
UpdateRequest updateRequest = new UpdateRequest("hotel", hotel.getId().toString());
updateRequest.doc(JSONUtil.toJsonStr(hotelDoc),XContentType.JSON);*/
UpdateRequest updateRequest = new UpdateRequest("hotel", "45845");
updateRequest.doc("score", "48"); // 直接按照kv的格式
UpdateResponse update = client.update(updateRequest, RequestOptions.DEFAULT);
}
/**
* 删除文档
*/
@Test
void testDeleteDocument() throws IOException {
DeleteRequest deleteRequest = new DeleteRequest("hotel", "45845");
System.out.println(client.delete(deleteRequest, RequestOptions.DEFAULT).getVersion());
}
/**
* 批量导入文档
*/
@SneakyThrows
@Test
void testBatchAddDoucument(){
// 查询所有记录并转换成HotelDoc
List<HotelDoc> hotelDocs = iHotelService.list().stream().map(hotel -> new HotelDoc(hotel)).collect(Collectors.toList());
BulkRequest bulkRequest = new BulkRequest("hotel");
for (HotelDoc hotelDoc : hotelDocs) {
bulkRequest.add(new IndexRequest("hotel")
.id(hotelDoc.getId().toString())
.source(JSONUtil.toJsonStr(hotelDoc),XContentType.JSON));
}
client.bulk(bulkRequest,RequestOptions.DEFAULT);
}
除了批量新增，同时可以批量修改删除
4、复杂查询 1、query下的查询 1、match_all 全查询DSL:
java rest:
@Test
@SneakyThrows
void testMatchAll(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.matchAllQuery());
SearchHit[] hits = client.search(searchRequest, RequestOptions.DEFAULT).getHits().getHits(); // 第二个hits才能获取到数据
List<HotelDoc> hotelDocs = Arrays.stream(hits).map(hit -> {
// 使用流提取其中的json转换成HotelDoc实体后添加进集合中
return JSONUtil.toBean(hit.getSourceAsString().toString(), HotelDoc.class);
}).collect(Collectors.toList());
hotelDocs.forEach(hotelDoc -> System.out.println(hotelDoc.getName()));
}
@Test
@SneakyThrows
void testMultipMatch(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.multiMatchQuery("如家","brand","name"));
handleResponse(searchRequest);
}
@SneakyThrows
@Test
void testTerm(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.termQuery("city","北京"));
handleResponse(searchRequest);
}
@SneakyThrows
@Test
void testTerm(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source()
.query(QueryBuilders.rangeQuery("price").lte(400));
handleResponse(searchRequest);
}
- must：必须匹配每个子查询，类似“与”
- should：选择性匹配子查询，类似“或”
- must_not：必须不匹配，不参与算分，类似“非”
- filter：必须匹配，不参与算分
@Test
@SneakyThrows
void testBooleanQuery(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.boolQuery()
.must(QueryBuilders.termQuery("brand","如家"))
.mustNot(QueryBuilders.rangeQuery("price").gte(500))
.filter(QueryBuilders.geoDistanceQuery("location").point(31,121 ).distance(50, DistanceUnit.KILOMETERS))
);
handleResponse(searchRequest);
}
@Test
@SneakyThrows
void testGeoDistance(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.geoDistanceQuery("location")
.distance(15,DistanceUnit.KILOMETERS).point(31.21,121.5));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
List<HotelDoc> hotelDocs = Arrays.stream(searchResponse.getHits().getHits()).map(hit -> {
return JSONUtil.toBean(hit.getSourceAsString().toString(), HotelDoc.class);
}).collect(Collectors.toList());
hotelDocs.forEach(hotelDoc -> {
System.out.println(hotelDoc.getName());
});
}
@Test
@SneakyThrows
void testFunctionScore(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().query(QueryBuilders.functionScoreQuery(
QueryBuilders.matchQuery("all","上海"),
new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{
new FunctionScoreQueryBuilder.FilterFunctionBuilder(
QueryBuilders.rangeQuery("price").gte(700),
ScoreFunctionBuilders.weightFactorFunction(10)
)
}
));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
List<HotelDoc> hotelDocs = Arrays.stream(searchResponse.getHits().getHits()).map(hit -> {
return JSONUtil.toBean(hit.getSourceAsString().toString(), HotelDoc.class);
}).collect(Collectors.toList());
hotelDocs.forEach(hotelDoc -> {
System.out.print("name:"+hotelDoc.getName());
System.out.println(" age:"+hotelDoc.getPrice());
});
}
@Test
@SneakyThrows
void testSortPageQuery(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source()
.query(QueryBuilders.termQuery("brand","如家"))
.from(10).size(1)
.sort("price", SortOrder.DESC).sort("score",SortOrder.ASC);
handleResponse(searchRequest);
}
注意：如果查询的字段不是高亮的字段，必须显示修改require_field_match的值为false
高亮的结果与查询的文档结果默认是分离的，并不在一起。因此解析高亮的代码需要额外的做处理
@Test
@SneakyThrows
void testHighlighter(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source()
.query(QueryBuilders.termQuery("name","如家"))
.highlighter(new HighlightBuilder().field("name").requireFieldMatch(false)
.preTags("").postTags("")
// 以最后一个标签为主
.field("brand").preTags("").postTags(""));
handleResponse(searchRequest);
}
private void handleResponse(SearchRequest searchRequest) throws IOException {
SearchHits searchHits = client.search(searchRequest, RequestOptions.DEFAULT).getHits();
System.out.printf("总共获取的条数为：%s", searchHits.getTotalHits().value+"\n");
SearchHit[] hits = searchHits.getHits();
List<HotelDoc> hotelDocs = Arrays.stream(hits).map(hit -> {
String highlightName = null;
String highlightBrand = null;
if (!CollectionUtils.isEmpty(hit.getHighlightFields())){
// 获取高亮字段
highlightName = hit.getHighlightFields().get("name").getFragments()[0].string();
highlightBrand = hit.getHighlightFields().get("brand").getFragments()[0].string();
}
HotelDoc hotelDoc = JSONUtil.toBean(hit.getSourceAsString().toString(), HotelDoc.class);
if (highlightName!=null){
// 将查询出来的高亮字段覆盖到原查询的实体中的字段
hotelDoc.setName(highlightName);
}
if (highlightBrand!=null){
hotelDoc.setBrand(highlightBrand);
}
return hotelDoc;})
.collect(Collectors.toList());
// hotelDocs.forEach(hotelDoc -> System.out.println("酒店名："+hotelDoc.getName()+" 酒店品牌："+hotelDoc.getBrand()));
hotelDocs.forEach(System.out::println);
}
聚合常见的有三类：
- **桶（Bucket）**聚合：用来对文档做分组一定不分词
- TermAggregation：按照文档字段值分组，例如按照品牌值分组、按照国家分组
- Date Histogram：按照日期阶梯分组，例如一周为一组，或者一月为一组
- **度量（Metric）**聚合：用以计算一些值，比如：最大值、最小值、平均值等
- Avg：求平均值
- Max：求最大值
- Min：求最小值
- Stats：同时求max、min、avg、sum等
- **管道（pipeline）**聚合：其它聚合的结果为基础做聚合
这里只展示前面两种
1、Bucket聚合&排序&范围品牌聚合，并且排序，限定聚合范围
@Test
@SneakyThrows
void testAggregationsBrand(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source()
.query(QueryBuilders.rangeQuery("price").gte(100).lte(500))
.aggregation(AggregationBuilders.terms("brand_agg") // 聚合字段名
.field("brand").order(BucketOrder.key(false)).size(10)
);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// 注意转换的类型为Terms
Terms brand_agg = (Terms) searchResponse.getAggregations().get("brand_agg");
for (Terms.Bucket bucket : brand_agg.getBuckets()) {
System.out.println(bucket.getKeyAsString());
}
}
前面的Bucket聚合对酒店按照品牌分组，形成了一个个桶。现在我们需要对桶内的酒店做运算，获取每个品牌的用户评分的min、max、avg等值。
这就要用到Metric聚合了，例如stat聚合：就可以获取min、max、avg等结果
@Test
@SneakyThrows
void testAggregationBrand_score(){
SearchRequest searchRequest = new SearchRequest("hotel");
searchRequest.source().aggregation(AggregationBuilders.terms("brand_agg")
.field("brand").size(20)
// 在对品牌做聚合的前提下做分数的聚合
.subAggregation(AggregationBuilders.stats("score_stats").field("score")));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
Terms terms = (Terms) searchResponse.getAggregations().get("brand_agg");
terms.getBuckets().forEach(bucket ->{
System.out.println("品牌："+bucket.getKeyAsString());
ParsedStats stats = (ParsedStats) bucket.getAggregations().get("score_stats");
System.out.printf("最大值为：%s ", stats.getMaxAsString());
System.out.printf("最小值为：%s ", stats.getMinAsString());
System.out.printf("平均值为：%s ", stats.getAvgAsString());
System.out.printf("和为：%s ", stats.getSumAsString());
System.out.printf("总数为：%s\n", stats.getCount());
} );
}
好啦，本文关于springboot整合es的主要api的 *** 作就介绍到这里啦，如果你对本文的内容有疑问或者其他方面的见解，欢迎到评论区留言
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）