elasticsearch之search template

無風聽海發表於2023-01-19

一、search template簡介

elasticsearch提供了search template功能,其會在實際執行查詢之前,對search template進行預處理並將引數填充到template中。

elasticsearch主要提供了兩個API來支援search template
_scripts/用於對search template的維護;
_search/template使用search template進行搜尋;

二、測試資料準備

批次index三個檔案

POST _bulk

{ "index" : { "_index" : "search_template_test", "_type" : "_doc", "_id" : "1" } }
{ "name":"zhang san",	"age":30,	"man":true,	"address":"Hebei LangFang"}
{ "index" : { "_index" : "search_template_test", "_type" : "_doc", "_id" : "2" } }
{ "name":"li si",	"age":20,	"man":true,	"address":"BeiJing HaiDian"}
{ "index" : { "_index" : "search_template_test", "_type" : "_doc", "_id" : "3" } }
{ "name":"liu hui",	"age":40,	"man":false,	"address":"NeiMengGu ChiFeng"}


查詢檢視三個索引的檔案

GET search_template_test/_search
{
    "took":6,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":3,
        "max_score":1,
        "hits":[
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"2",
                "_score":1,
                "_source":{
                    "name":"li si",
                    "age":20,
                    "man":true,
                    "address":"BeiJing HaiDian"
                }
            },
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"1",
                "_score":1,
                "_source":{
                    "name":"zhang san",
                    "age":30,
                    "man":true,
                    "address":"Hebei LangFang"
                }
            },
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"3",
                "_score":1,
                "_source":{
                    "name":"liu hui",
                    "age":40,
                    "man":false,
                    "address":"NeiMengGu ChiFeng"
                }
            }
        ]
    }
}

三、維護search template

我們新建id為search_template_test.match_name的search template,其主要是查詢match 欄位name;

POST _scripts/search_template_test.match_name
{
    "script": {        
        "lang": "mustache",
        "source": {
            "query": {
                "match": {
                    "name": "{{name_val}}"
                }
            }
        }
    }
}

檢視我們建立的search template

GET _scripts/search_template_test.match_name

{
    "_id":"search_template_test.match_name",
    "found":true,
    "script":{
        "lang":"mustache",
        "source":"{\"query\":{\"match\":{\"name\":\"{{name_val}}\"}}}",
        "options":{
            "content_type":"application/json; charset=UTF-8"
        }
    }
}

刪除我們建立的search template

DELET _scripts/search_template_test.match_name
{
    "acknowledged":true
}

elasticsearch提供了API支援我們檢視預處理最終形成的查詢語句;

POST _render/template/search_template_test.match_name
{
    "params":{
        "name_val":"zhang li"
    }
}


{
    "template_output":{
        "query":{
            "match":{
                "name":"zhang li"
            }
        }
    }
}

四、使用search template

我們可以像下邊這樣使用search template

POST search_template_test/_search/template
{
    "id": "search_template_test.match_name", 
    "params": {
        "name_val":"zhang li"
    }
}



{
    "took":1,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":2,
        "max_score":0.2876821,
        "hits":[
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"2",
                "_score":0.2876821,
                "_source":{
                    "name":"li si",
                    "age":20,
                    "man":true,
                    "address":"BeiJing HaiDian"
                }
            },
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"1",
                "_score":0.2876821,
                "_source":{
                    "name":"zhang san",
                    "age":30,
                    "man":true,
                    "address":"Hebei LangFang"
                }
            }
        ]
    }
}

執行search template也支援使用explain來除錯檢視elasticsearch的打分情況;

POST search_template_test/_search/template
{
    "id": "search_template_test.match_name", 
    "params": {
        "name_val":"zhang li"
    },
    "explain":true
}




{
    "took":3,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":2,
        "max_score":0.2876821,
        "hits":[
            {
                "_shard":"[search_template_test][2]",
                "_node":"Sl6S4Kn2Rh-BdNgaM3ZwJg",
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"2",
                "_score":0.2876821,
                "_source":{
                    "name":"li si",
                    "age":20,
                    "man":true,
                    "address":"BeiJing HaiDian"
                },
                "_explanation":{
                    "value":0.2876821,
                    "description":"sum of:",
                    "details":[
                        {
                            "value":0.2876821,
                            "description":"weight(name:li in 0) [PerFieldSimilarity], result of:",
                            "details":[
                                ......
                            ]
                        }
                    ]
                }
            },
            {
                "_shard":"[search_template_test][3]",
                "_node":"Sl6S4Kn2Rh-BdNgaM3ZwJg",
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"1",
                "_score":0.2876821,
                "_source":{
                    "name":"zhang san",
                    "age":30,
                    "man":true,
                    "address":"Hebei LangFang"
                },
                "_explanation":{
                    "value":0.2876821,
                    "description":"sum of:",
                    "details":[
                        {
                            "value":0.2876821,
                            "description":"weight(name:zhang in 0) [PerFieldSimilarity], result of:",
                            "details":[
                                ......
                            ]
                        }
                    ]
                }
            }
        ]
    }
}

執行search template也支援使用profile來除錯檢視elasticsearch的查詢執行情況;

POST search_template_test/_search/template
{
    "id": "search_template_test.match_name", 
    "params": {
        "name_val":"zhang li"
    },
    "profile":true
}



{
    "took":11,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":2,
        "max_score":0.2876821,
        "hits":[

        ]
    },
    "profile":{
        "shards":[
            {
                "id":"[Sl6S4Kn2Rh-BdNgaM3ZwJg][search_template_test][0]",
                "searches":[
                    {
                        "query":[
                            {
                                "type":"BooleanQuery",
                                "description":"name:zhang name:li",
                                "time_in_nanos":308301,
                                "breakdown":{
                                    "score":0,
                                    "build_scorer_count":0,
                                    "match_count":0,
                                    "create_weight":308300,
                                    "next_doc":0,
                                    "match":0,
                                    "create_weight_count":1,
                                    "next_doc_count":0,
                                    "score_count":0,
                                    "build_scorer":0,
                                    "advance":0,
                                    "advance_count":0
                                },
                                "children":[
                                    {
                                        "type":"TermQuery",
                                        "description":"name:zhang",
                                        "time_in_nanos":32901,
                                        "breakdown":{
                                            "score":0,
                                            "build_scorer_count":0,
                                            "match_count":0,
                                            "create_weight":32900,
                                            "next_doc":0,
                                            "match":0,
                                            "create_weight_count":1,
                                            "next_doc_count":0,
                                            "score_count":0,
                                            "build_scorer":0,
                                            "advance":0,
                                            "advance_count":0
                                        }
                                    },
                                    {
                                        "type":"TermQuery",
                                        "description":"name:li",
                                        "time_in_nanos":15901,
                                        "breakdown":{
                                            "score":0,
                                            "build_scorer_count":0,
                                            "match_count":0,
                                            "create_weight":15900,
                                            "next_doc":0,
                                            "match":0,
                                            "create_weight_count":1,
                                            "next_doc_count":0,
                                            "score_count":0,
                                            "build_scorer":0,
                                            "advance":0,
                                            "advance_count":0
                                        }
                                    }
                                ]
                            }
                        ],
                        "rewrite_time":42700,
                        "collector":[
                            {
                                "name":"CancellableCollector",
                                "reason":"search_cancelled",
                                "time_in_nanos":700,
                                "children":[
                                    {
                                        "name":"SimpleTopScoreDocCollector",
                                        "reason":"search_top_hits",
                                        "time_in_nanos":200
                                    }
                                ]
                            }
                        ]
                    }
                ],
                "aggregations":[

                ]
            }
        ]
    }
}

五、使用search template處理複雜的查詢

我們構建以下search template,其功能如下

可以透過name欄位進行match,如果命中欄位中的各個關鍵字相鄰則有更高的權重打分,同時可以透過address欄位進行模糊匹配,這三個查詢只要命中一個即可;

man欄位值必須符合輸入的值;

POST _scripts/search_template_test.multi_search_template
{
    "script":{
        "lang":"mustache",
        "source":{
            "query":{
                "bool":{
                    "should":[
                        {
                            "match":{
                                "name":"{{name}}"
                            }
                        },
                        {
                            "match_phrase_prefix":{
                                "name":"{{name}}"
                            }
                        },
                        {
                            "query_string":{
                                "default_field":"address",
                                "query":"{{#join delimiter=' '}}address{{/join delimiter=' '}}"
                            }
                        }
                    ],
                    "filter":{
                        "term":{
                            "man":"{{man}}"
                        }
                    },
                    "minimum_should_match":1
                }
            }
        }
    }
}

透過以下引數命中前兩條資料

POST search_template_test/_search/template
{
    "id":"search_template_test.multi_search_template",
    "params":{
        "name":"zhang sa",
        "address":[
            "*bei*",
            "*chi*"
        ],
        "man":true
    }
}


{
    "took":10,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":2,
        "max_score":1.8630463,
        "hits":[
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"1",
                "_score":1.8630463,
                "_source":{
                    "name":"zhang san",
                    "age":30,
                    "man":true,
                    "address":"Hebei LangFang"
                }
            },
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"2",
                "_score":1,
                "_source":{
                    "name":"li si",
                    "age":20,
                    "man":true,
                    "address":"BeiJing HaiDian"
                }
            }
        ]
    }
}

我們將man欄位設定為false,從而只匹配第三條記錄;

POST search_template_test/_search/template
{
    "id":"search_template_test.multi_search_template",
    "params":{
        "name":"zhang sa",
        "address":[
            "*bei*",
            "*chi*"
        ],
        "man":false
    }
}

{
    "took":4,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_index":"search_template_test",
                "_type":"_doc",
                "_id":"3",
                "_score":1,
                "_source":{
                    "name":"liu hui",
                    "age":40,
                    "man":false,
                    "address":"NeiMengGu ChiFeng"
                }
            }
        ]
    }
}

相關文章