Java操作elasticSearch复杂查询以及解析数据以及索引保存数据-摩杜云开发者社区

part1

一、Java操作elasticSearch复杂查询以及解析数据

1.1 拆解操作数据
1.2 构造一个查询器指向索引
1.3 封装查询条件器
1.4 封装的条件器置入查询器
1.5 容器中的client调用查询：
1.6 解析查询结果
1.7 打印逐条记录时，可以把结构封装成一个model 借助一下：json.cn
1.8 完整操作：

二、Java操作elasticSearch索引保存数据

2.1 计划与实现
2.2单元测试
2.3 执行
2.4 查看一下Kibana操作结果

一、Java操作elasticSearch复杂查询以及解析数据

说明：基于银行测试库的操作

es的银行测试库，看一个Kibana操作然后用java检索解析这个数据

聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均薪资

#聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均薪资
GET bank/_search
{
  "query":{
    "match": {
      "address": "mill"
    }
  },
  "aggs": {
    "ageAgg": {
      "terms": {
        "field": "age",
        "size": 10
      }
    },
    "balanceAvg":{
      "avg":{
        "field": "balance"
      }
    }
  },
  "size": 0
}

以下是分解思路实现步骤：

1.1 拆解操作数据

#聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均年龄
GET bank/_search
{
“query”:{ “match”: { “address”: “mill” }
},
“aggs”: { “ageAgg”: { “terms”: { “field”: “age”, “size”: 10 } },

“balanceAvg”:{ “avg”:{ “field”: “balance” } } }, “size”: 0 }

1.2 构造一个查询器指向索引

SearchRequest searchRequest = new SearchRequest();
//指定索引
searchRequest.indices("bank");

1.3 封装查询条件器

//指定DSL 检索条件
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//构造检索条件
searchSourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
//按照年龄只分布进行聚合
TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
searchSourceBuilder.aggregation(ageAgg);
//计算平均薪资
AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("blance");
searchSourceBuilder.aggregation(balanceAvg);

//打印检索条件 打印结果与Kibana核对
System.out.println("检索条件："+searchSourceBuilder);

检索条件：{"query":{"match":{"address":{"query":"mill","operator":"OR","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"boost":1.0}}},"aggregations":{"ageAgg":{"terms":{"field":"age","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]}},"balanceAvg":{"avg":{"field":"blance"}}}}

1.4 封装的条件器置入查询器

searchRequest.source(searchSourceBuilder);

1.5 容器中的client调用查询：

//执行检索
SearchResponse search = client.search(searchRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);

1.6 解析查询结果

System.out.println(search.toString());
//		Map map = JSON.parseObject(search.toString(), Map.class);
		//分析结果 查询结构
		SearchHits hits = search.getHits();
		SearchHit[] searchHits = hits.getHits();
		for (SearchHit hit: searchHits){
//			hit.getIndex();
//			hit.getId();
			String sourceAsString = hit.getSourceAsString();
			Accout accout = JSON.parseObject(sourceAsString, Accout.class);
			System.out.println(accout.toString());
		}
		//获取检索的分析信息
		Aggregations aggregations = search.getAggregations();
//		for (Aggregation aggregation : aggregations.asList()) {
//			System.out.println("当前聚合名字："+aggregation.getName());
//		}
		//分类聚合
		Terms ageAgg1 = aggregations.get("ageAgg");
		for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
			String keyAsString = bucket.getKeyAsString();
			System.out.println("年龄：" + keyAsString + "人数："+bucket.getDocCount());
		}
		//平局值
		Avg balanceAvg1 = aggregations.get("balanceAvg");
		System.out.println("平均薪资"+ balanceAvg1.getValue());

{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":4,"relation":"eq"},"max_score":5.4032025,"hits":[{"_index":"bank","_type":"account","_id":"970","_score":5.4032025,"_source":{"account_number":970,"balance":19648,"firstname":"Forbes","lastname":"Wallace","age":28,"gender":"M","address":"990 Mill Road","employer":"Pheast","email":"forbeswallace@pheast.com","city":"Lopezo","state":"AK"}},{"_index":"bank","_type":"account","_id":"136","_score":5.4032025,"_source":{"account_number":136,"balance":45801,"firstname":"Winnie","lastname":"Holland","age":38,"gender":"M","address":"198 Mill Lane","employer":"Neteria","email":"winnieholland@neteria.com","city":"Urie","state":"IL"}},{"_index":"bank","_type":"account","_id":"345","_score":5.4032025,"_source":{"account_number":345,"balance":9812,"firstname":"Parker","lastname":"Hines","age":38,"gender":"M","address":"715 Mill Avenue","employer":"Baluba","email":"parkerhines@baluba.com","city":"Blackgum","state":"KY"}},{"_index":"bank","_type":"account","_id":"472","_score":5.4032025,"_source":{"account_number":472,"balance":25571,"firstname":"Lee","lastname":"Long","age":32,"gender":"F","address":"288 Mill Street","employer":"Comverges","email":"leelong@comverges.com","city":"Movico","state":"MT"}}]},"aggregations":{"lterms#ageAgg":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":38,"doc_count":2},{"key":28,"doc_count":1},{"key":32,"doc_count":1}]},"avg#balanceAvg":{"value":null}}}
GulimallSearchApplicationTests.Accout(account_number=970, balance=19648, firstname=Forbes, lastname=Wallace, age=28, gender=M, address=990 Mill Road, employer=Pheast, email=forbeswallace@pheast.com, city=Lopezo, state=AK)
GulimallSearchApplicationTests.Accout(account_number=136, balance=45801, firstname=Winnie, lastname=Holland, age=38, gender=M, address=198 Mill Lane, employer=Neteria, email=winnieholland@neteria.com, city=Urie, state=IL)
GulimallSearchApplicationTests.Accout(account_number=345, balance=9812, firstname=Parker, lastname=Hines, age=38, gender=M, address=715 Mill Avenue, employer=Baluba, email=parkerhines@baluba.com, city=Blackgum, state=KY)
GulimallSearchApplicationTests.Accout(account_number=472, balance=25571, firstname=Lee, lastname=Long, age=32, gender=F, address=288 Mill Street, employer=Comverges, email=leelong@comverges.com, city=Movico, state=MT)
年龄：38人数：2
年龄：28人数：1
年龄：32人数：1
平均薪资25208.0

1.7 打印逐条记录时，可以把结构封装成一个model 借助一下：json.cn

Java操作elasticSearch复杂查询以及解析数据以及索引保存数据_System

Java操作elasticSearch复杂查询以及解析数据以及索引保存数据_开发语言_02

1.8 完整操作：

@ToString
		@Data
		static class Accout {

			private int account_number;
			private int balance;
			private String firstname;
			private String lastname;
			private int age;
			private String gender;
			private String address;
			private String employer;
			private String email;
			private String city;
			private String state;
		}

	@Test
	public void searchData() throws IOException {
		SearchRequest searchRequest = new SearchRequest();
		//指定索引
		searchRequest.indices("bank");
		//指定DSL 检索条件
		SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
		//构造检索条件

	/**
	 #聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均年龄
	 GET bank/_search
	 {
	 "query":{ "match": { "address": "mill" }
	 },
	 "aggs": { "ageAgg": { "terms": { "field": "age", "size": 10 } },
	 "balanceAvg":{ "avg":{ "field": "balance" } } }, "size": 0 }
	 */

//		searchSourceBuilder.aggregation();
//		searchSourceBuilder.from();
//		searchSourceBuilder.size();
		searchSourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
		//按照年龄只分布进行聚合
		TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
		searchSourceBuilder.aggregation(ageAgg);
		//计算平均薪资
		AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
		searchSourceBuilder.aggregation(balanceAvg);

		//打印检索条件
 		System.out.println("检索条件："+searchSourceBuilder);


		searchRequest.source(searchSourceBuilder);
		//执行检索
		SearchResponse search = client.search(searchRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);

		//分析结果
//		searchRequest.
		System.out.println(search.toString());
//		Map map = JSON.parseObject(search.toString(), Map.class);
		//分析结果 查询结构
		SearchHits hits = search.getHits();
		SearchHit[] searchHits = hits.getHits();
		for (SearchHit hit: searchHits){
//			hit.getIndex();
//			hit.getId();
			String sourceAsString = hit.getSourceAsString();
			Accout accout = JSON.parseObject(sourceAsString, Accout.class);
			System.out.println(accout.toString());
		}
		//获取检索的分析信息
		Aggregations aggregations = search.getAggregations();
//		for (Aggregation aggregation : aggregations.asList()) {
//			System.out.println("当前聚合名字："+aggregation.getName());
//		}
		//分类聚合
		Terms ageAgg1 = aggregations.get("ageAgg");
		for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
			String keyAsString = bucket.getKeyAsString();
			System.out.println("年龄：" + keyAsString + "人数："+bucket.getDocCount());
		}
		//平局值
		Avg balanceAvg1 = aggregations.get("balanceAvg");
		System.out.println("平均薪资"+ balanceAvg1.getValue());

	}

part2

一、Java操作elasticSearch复杂查询以及解析数据

1.1 拆解操作数据
1.2 构造一个查询器指向索引
1.3 封装查询条件器
1.4 封装的条件器置入查询器
1.5 容器中的client调用查询：
1.6 解析查询结果
1.7 打印逐条记录时，可以把结构封装成一个model 借助一下：json.cn
1.8 完整操作：

二、Java操作elasticSearch索引保存数据

2.1 计划与实现
2.2单元测试
2.3 执行
2.4 查看一下Kibana操作结果

二、Java操作elasticSearch索引保存数据

2.1 计划与实现

存储一个新索引students，然后保存文档

借助Kibana:

> GET /students/_search

结果：
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [students]",
        "resource.type" : "index_or_alias",
        "resource.id" : "students",
        "index_uuid" : "_na_",
        "index" : "students"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [students]",
    "resource.type" : "index_or_alias",
    "resource.id" : "students",
    "index_uuid" : "_na_",
    "index" : "students"
  },
  "status" : 404
}

索引不存在

2.2单元测试

索引数据的请求是个网络操作，所以会有异常处理。

//做一个学生对象
	//注解后setter getter
	@Data
	class Student{
		private String name;
		private Integer age;
		private String gender;
	}

	@Test
	public void indexData() throws IOException {
		//索引
		IndexRequest indexRequest = new IndexRequest("students");
		//数据id 不设置会自动生成
		indexRequest.id("1");

		Student student = new Student();
		student.setAge(18);
		student.setGender("男");
		student.setName("张铁蛋");
		//对象转换json
		String jsonString = JSON.toJSONString(student);
		//索引对象加入对象json  声明保存形式
		indexRequest.source(jsonString, XContentType.JSON);
		//用容器中导入的client 调用请求  索引对象 和 配置参数 这个配置参数是整合配置时搞定的
		IndexResponse index = client.index(indexRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);
		//index为相应数据
		System.out.println(index);
	}

2.3 执行

2021-11-05 16:01:28.219  INFO 1548 --- [           main] c.a.g.s.GulimallSearchApplicationTests   : Started GulimallSearchApplicationTests in 18.417 seconds (JVM running for 19.985)
IndexResponse[index=students,type=_doc,id=1,version=1,result=created,seqNo=0,primaryTerm=1,shards={"total":2,"successful":1,"failed":0}]
2021-11-05 16:01:32.019  INFO 1548 --- [       Thread-9] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'

Process finished with exit code 0

2.4 查看一下Kibana操作结果

GET /students/_search

结果集：

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "students",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "age" : 18,
          "gender" : "男",
          "name" : "张铁蛋"
        }
      }
    ]
  }
}

至此，保存成功

part1

一、Java操作elasticSearch复杂查询以及解析数据

1.1 拆解操作数据

1.2 构造一个查询器 指向索引

1.3 封装查询条件器

1.4 封装的条件器置入查询器

1.5 容器中的client调用查询：

1.6 解析查询结果

1.7 打印逐条记录时，可以把结构封装成一个model 借助一下：json.cn

1.8 完整操作：

part2

二、Java操作elasticSearch索引保存数据

2.1 计划与实现

2.2单元测试

2.3 执行

2.4 查看一下Kibana操作结果

1.2 构造一个查询器指向索引