Kibana查詢耗時

滄南發表於2017-03-01

kibana 查詢,會對查詢時間段,做聚合操作,最近15min,則會聚合每30秒的文件數:
這裡寫圖片描述


當查詢時間範圍內的文件數特別大時,將非常耗時,有時會報錯:
這裡寫圖片描述

查詢Request Payload:

{"index":"www_log-*","ignore_unavailable":true}
{"size":500,"sort":[{"@timestamp":{"order":"desc","unmapped_type":"boolean"}}],"query":{"filtered":{"query":{"query_string":{"analyze_wildcard":true,"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1449469826877,"lte":1449470726877}}}],"must_not":[]}}}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}},"fragment_size":2147483647},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"30s","pre_zone":"+08:00","pre_zone_adjust_large_interval":true,"min_doc_count":0,"extended_bounds":{"min":1449469826876,"max":1449470726876}}}},"fields":["*","_source"],"script_fields":{},"fielddata_fields":[“@timestamp”]}

如果沒有時間格式欄位:

{"index":"www_log-*","ignore_unavailable":true}
{"size":500,"sort":[{"_score":{"order":"desc","unmapped_type":"boolean"}}],"query":{"query_string":{"analyze_wildcard":true,"query":"*"}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}},"fragment_size":2147483647},"fields":["*","_source"],"script_fields":{},"fielddata_fields":[]}

對索引www_log-*,2015-12-07 14:30:00 至 2015-12-07 14:35:00的5min資料,做聚合操作:

http://10.1.***:9200/www_log-*/logs/
{
  "query": {
"filtered": {
  "filter": {
    "range": {
      "request": {
        "gte": 1449469826877,
        "lte": 1449470126877
      }
    }
  }
}
  },
 "aggs": {
"request": {
  "date_histogram": {
    "field": "request",
    "interval": "30s",
    "pre_zone": "+08:00",
    "min_doc_count": 0
  }
}
  }
}

耗時199秒。

查詢aguid=1447075650199_3724624的資料,並且對2015-12-07 14:30:00 至 2015-12-07 14:45:00的15min資料,做聚合操作,耗時224秒,雖然聚合結果為0。

{
  "query": {
"filtered": {
  "query": {
    "query_string": {
      "analyze_wildcard": true,
      "query": "aguid:\"1447075650199_3724624\""
    }
  },
  "filter": {
    "range": {
      "request": {
        "gte": 1441469826877,
        "lte": 1449470726877
      }
    }
  }
}
  },
  "aggs": {
"request": {
  "date_histogram": {
    "field": "request",
    "interval": "30s",
    "pre_zone": "+08:00",
    "min_doc_count": 0,
    "extended_bounds": {
      "min": 1449469826876,
      "max": 1449470726876
    }
  }
}
}

去掉聚合操作,耗時14秒。
這裡寫圖片描述

相關文章