Elasticsearch 系列(五)- 数据聚合

  • Elasticsearch 系列(五)- 数据聚合已关闭评论
  • 15 次浏览
  • A+
所属分类:.NET技术
摘要

本章将和大家分享 Elasticsearch 中的数据聚合功能,通过聚合(aggregations)可以实现对文档数据的统计、分析、运算。

本章将和大家分享 Elasticsearch 中的数据聚合功能,通过聚合(aggregations)可以实现对文档数据的统计、分析、运算。

一、数据聚合-聚合的分类

聚合(aggregations)可以实现对文档数据的统计、分析、运算。聚合的官方文档地址:https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html

聚合常见的有三类:

1)桶(Bucket)聚合:用来对文档做分组

  • TermAggregation:按照文档字段值分组
  • Date Histogram:按照日期阶梯分组,例如:一周为一组,或者一月为一组。

2)度量(Metric)聚合:用于计算一些值,比如:最大值、最小值、平均值等。

  • Avg:求平均值
  • Max:求最大值
  • Min:求最小值
  • Stats:同时求max、min、avg、sum等。

3)管道(Pipeline)聚合:以其它聚合的结果为基础做聚合。

总结:

1)什么是聚合?

  • 聚合是对文档数据的统计、分析、计算

2)聚合的常见种类有哪些?

  • Bucket:对文档数据分组,并统计每组数量
  • Metric:对文档数据做计算,例如:avg
  • Pipeline:基于其它聚合结果再做聚合

3)参与聚合的字段类型不能是 text(可分词的文本)类型,可以是:keyword、数值、日期、布尔类型。

二、数据聚合-DSL实现Bucket聚合

1、DSL实现Bucket聚合

现在,我们要统计所有数据中的酒店品牌有几种,此时可以根据酒店品牌的名称做聚合。

类型为term类型,DSL示例:

# 聚合功能 GET /hotel/_search {   "size": 0, //设置size为0,结果中不包含文档,只包含聚合结果   "aggs": { //定义聚合     "brandAgg": { //给聚合起个名字       "terms": { //聚合的类型,按照品牌值聚合,所以选择term         "field": "brand", //参与聚合的字段         "size": 10 //希望获取的聚合结果数量       }     }   } }

运行结果如下:

Elasticsearch 系列(五)- 数据聚合

{   "took" : 1,   "timed_out" : false,   "_shards" : {     "total" : 1,     "successful" : 1,     "skipped" : 0,     "failed" : 0   },   "hits" : {     "total" : {       "value" : 201,       "relation" : "eq"     },     "max_score" : null,     "hits" : [ ]   },   "aggregations" : {     "brandAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 39,       "buckets" : [         {           "key" : "7天酒店",           "doc_count" : 30         },         {           "key" : "如家",           "doc_count" : 30         },         {           "key" : "皇冠假日",           "doc_count" : 17         },         {           "key" : "速8",           "doc_count" : 15         },         {           "key" : "万怡",           "doc_count" : 13         },         {           "key" : "华美达",           "doc_count" : 13         },         {           "key" : "和颐",           "doc_count" : 12         },         {           "key" : "万豪",           "doc_count" : 11         },         {           "key" : "喜来登",           "doc_count" : 11         },         {           "key" : "希尔顿",           "doc_count" : 10         }       ]     }   } }

2、Bucket聚合-聚合结果排序

默认情况下,Bucket聚合会统计Bucket内的文档数量,记为_count,并且按照_count降序排序。

我们可以修改结果排序方式:

# 聚合功能,自定义排序规则 GET /hotel/_search {   "size": 0,   "aggs": {     "brandAgg": {       "terms": {         "field": "brand",         "size": 10,         "order": {           "_count": "asc" //按照_count升序排列         }       }     }   } }

运行结果如下:

{   "took" : 0,   "timed_out" : false,   "_shards" : {     "total" : 1,     "successful" : 1,     "skipped" : 0,     "failed" : 0   },   "hits" : {     "total" : {       "value" : 201,       "relation" : "eq"     },     "max_score" : null,     "hits" : [ ]   },   "aggregations" : {     "brandAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 130,       "buckets" : [         {           "key" : "万丽",           "doc_count" : 2         },         {           "key" : "丽笙",           "doc_count" : 2         },         {           "key" : "君悦",           "doc_count" : 4         },         {           "key" : "豪生",           "doc_count" : 6         },         {           "key" : "维也纳",           "doc_count" : 7         },         {           "key" : "凯悦",           "doc_count" : 8         },         {           "key" : "希尔顿",           "doc_count" : 10         },         {           "key" : "汉庭",           "doc_count" : 10         },         {           "key" : "万豪",           "doc_count" : 11         },         {           "key" : "喜来登",           "doc_count" : 11         }       ]     }   } }

3、Bucket聚合-限定聚合范围

默认情况下,Bucket聚合是对索引库的所有文档做聚合,我们可以限定要聚合的文档范围,只要添加query条件即可。

示例:

# 聚合功能,限定聚合范围 GET /hotel/_search {   "query": {     "range": {       "price": {         "lte": 200 //只对200元以下的文档聚合       }     }   },   "size": 0,   "aggs": {     "brandAgg": {       "terms": {         "field": "brand",         "size": 10,         "order": {           "_count": "asc"         }       }     }   } }

运行结果如下:

{   "took" : 0,   "timed_out" : false,   "_shards" : {     "total" : 1,     "successful" : 1,     "skipped" : 0,     "failed" : 0   },   "hits" : {     "total" : {       "value" : 17,       "relation" : "eq"     },     "max_score" : null,     "hits" : [ ]   },   "aggregations" : {     "brandAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 0,       "buckets" : [         {           "key" : "7天酒店",           "doc_count" : 1         },         {           "key" : "汉庭",           "doc_count" : 1         },         {           "key" : "速8",           "doc_count" : 2         },         {           "key" : "如家",           "doc_count" : 13         }       ]     }   } }

4、总结

1)aggs代表聚合,与query同级,此时query的作用是?

  • 限定聚合的的文档范围

2)聚合必须的三要素是什么?

  • 聚合名称
  • 聚合类型
  • 聚合字段

3)聚合可配置属性有哪些?

  • size:指定聚合结果数量
  • order:指定聚合结果排序方式
  • field:指定聚合字段

三、数据聚合-DSL实现Metric聚合

例如:我们要求获取每个品牌的用户评分的min、max、avg等值。

我们可以利用stats聚合:

# 嵌套聚合Metric GET /hotel/_search {   "size": 0,   "aggs": {     "brandAgg": {       "terms": {         "field": "brand",         "size": 10,         "order": {           "scoreAgg.avg": "desc" //对桶里面的数据做排序         }       },       "aggs": { //是brandAgg聚合的子聚合,也就是分组后对每组分别计算         "scoreAgg": { //聚合名称           "stats": { //聚合类型,这里stats可以计算min、max、avg等             "field": "score" //聚合字段,这里是score           }         }       }     }   } }

运行结果如下所示:

Elasticsearch 系列(五)- 数据聚合

{   "took" : 0,   "timed_out" : false,   "_shards" : {     "total" : 1,     "successful" : 1,     "skipped" : 0,     "failed" : 0   },   "hits" : {     "total" : {       "value" : 201,       "relation" : "eq"     },     "max_score" : null,     "hits" : [ ]   },   "aggregations" : {     "brandAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 111,       "buckets" : [         {           "key" : "万丽",           "doc_count" : 2,           "scoreAgg" : {             "count" : 2,             "min" : 46.0,             "max" : 47.0,             "avg" : 46.5,             "sum" : 93.0           }         },         {           "key" : "凯悦",           "doc_count" : 8,           "scoreAgg" : {             "count" : 8,             "min" : 45.0,             "max" : 47.0,             "avg" : 46.25,             "sum" : 370.0           }         },         {           "key" : "和颐",           "doc_count" : 12,           "scoreAgg" : {             "count" : 12,             "min" : 44.0,             "max" : 47.0,             "avg" : 46.083333333333336,             "sum" : 553.0           }         },         {           "key" : "丽笙",           "doc_count" : 2,           "scoreAgg" : {             "count" : 2,             "min" : 46.0,             "max" : 46.0,             "avg" : 46.0,             "sum" : 92.0           }         },         {           "key" : "喜来登",           "doc_count" : 11,           "scoreAgg" : {             "count" : 11,             "min" : 44.0,             "max" : 48.0,             "avg" : 46.0,             "sum" : 506.0           }         },         {           "key" : "皇冠假日",           "doc_count" : 17,           "scoreAgg" : {             "count" : 17,             "min" : 44.0,             "max" : 48.0,             "avg" : 46.0,             "sum" : 782.0           }         },         {           "key" : "万豪",           "doc_count" : 11,           "scoreAgg" : {             "count" : 11,             "min" : 43.0,             "max" : 47.0,             "avg" : 45.81818181818182,             "sum" : 504.0           }         },         {           "key" : "万怡",           "doc_count" : 13,           "scoreAgg" : {             "count" : 13,             "min" : 44.0,             "max" : 48.0,             "avg" : 45.69230769230769,             "sum" : 594.0           }         },         {           "key" : "君悦",           "doc_count" : 4,           "scoreAgg" : {             "count" : 4,             "min" : 44.0,             "max" : 47.0,             "avg" : 45.5,             "sum" : 182.0           }         },         {           "key" : "希尔顿",           "doc_count" : 10,           "scoreAgg" : {             "count" : 10,             "min" : 37.0,             "max" : 48.0,             "avg" : 45.4,             "sum" : 454.0           }         }       ]     }   } }

四、数据聚合-多条件聚合

需求:搜索页面中的城市、星级、品牌等信息不应该是在页面写死,而是通过聚合索引库中的酒店数据得来的。

Elasticsearch 系列(五)- 数据聚合

示例:

# 多条件聚合 GET /hotel/_search {   "query": {     "bool": {       "must": [         {           "match": {             "all": "酒店"           }         }       ],       "should": [         {           "term": {             "brand": "皇冠假日"           }         },         {           "term": {             "brand": "华美达"           }         }       ],       "must_not": [         {           "range": {             "price": {               "lte": 500             }           }         }       ],       "filter": [         {           "range": {             "score": {               "gte": 45             }           }         }       ],       "minimum_should_match": 1,       "boost": 1     }   },   "size": 0,   "aggs": {     "cityAgg": {       "terms": {         "field": "city",         "size": 10,         "order": {           "_count": "desc"         }       }     },     "starNameAgg": {       "terms": {         "field": "starName",         "size": 10,         "order": {           "_count": "desc"         }       }     },     "brandAgg": {       "terms": {         "field": "brand",         "size": 10,         "order": {           "scoreAgg.avg": "desc"         }       },       "aggs": {         "scoreAgg": {           "stats": {             "field": "score"           }         }       }     }   } }

运行结果如下:

{   "took" : 1,   "timed_out" : false,   "_shards" : {     "total" : 1,     "successful" : 1,     "skipped" : 0,     "failed" : 0   },   "hits" : {     "total" : {       "value" : 17,       "relation" : "eq"     },     "max_score" : null,     "hits" : [ ]   },   "aggregations" : {     "brandAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 0,       "buckets" : [         {           "key" : "皇冠假日",           "doc_count" : 12,           "scoreAgg" : {             "count" : 12,             "min" : 45.0,             "max" : 48.0,             "avg" : 46.416666666666664,             "sum" : 557.0           }         },         {           "key" : "华美达",           "doc_count" : 5,           "scoreAgg" : {             "count" : 5,             "min" : 45.0,             "max" : 46.0,             "avg" : 45.2,             "sum" : 226.0           }         }       ]     },     "starNameAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 0,       "buckets" : [         {           "key" : "五钻",           "doc_count" : 7         },         {           "key" : "五星级",           "doc_count" : 5         },         {           "key" : "四星级",           "doc_count" : 3         },         {           "key" : "四钻",           "doc_count" : 2         }       ]     },     "cityAgg" : {       "doc_count_error_upper_bound" : 0,       "sum_other_doc_count" : 0,       "buckets" : [         {           "key" : "上海",           "doc_count" : 10         },         {           "key" : "北京",           "doc_count" : 4         },         {           "key" : "深圳",           "doc_count" : 3         }       ]     }   } }

至此本文就全部介绍完了,如果觉得对您有所启发请记得点个赞哦!!!

 

此文由博主精心撰写转载请保留此原文链接:https://www.cnblogs.com/xyh9039/p/18093166

版权声明:如有雷同纯属巧合,如有侵权请及时联系本人修改,谢谢!!!