ElasticSearch(四) ElasticSearch中文分詞外掛IK的簡單測試

OldBoy~發表於2018-01-11
Elasticsearch中文分詞

先來一個簡單的測試

# curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=standard&pretty" -d 'PHP是世界上最好的語言'   //_analyze表示分析分詞；analyzer=standard，表示分詞方式standard; -d表示測試的一段文字

測試結果

{
  "tokens" : [
    {
      "token" : "php",
      "start_offset" : 0,
      "end_offset" : 3,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "是",
      "start_offset" : 3,
      "end_offset" : 4,
      "type" : "<IDEOGRAPHIC>",
      "position" : 1
    },
    {
      "token" : "世",
      "start_offset" : 4,
      "end_offset" : 5,
      "type" : "<IDEOGRAPHIC>",
      "position" : 2
    },
    {
      "token" : "界",
      "start_offset" : 5,
      "end_offset" : 6,
      "type" : "<IDEOGRAPHIC>",
      "position" : 3
    },
    {
      "token" : "上",
      "start_offset" : 6,
      "end_offset" : 7,
      "type" : "<IDEOGRAPHIC>",
      "position" : 4
    },
    {
      "token" : "最",
      "start_offset" : 7,
      "end_offset" : 8,
      "type" : "<IDEOGRAPHIC>",
      "position" : 5
    },
    {
      "token" : "好",
      "start_offset" : 8,
      "end_offset" : 9,
      "type" : "<IDEOGRAPHIC>",
      "position" : 6
    },
    {
      "token" : "的",
      "start_offset" : 9,
      "end_offset" : 10,
      "type" : "<IDEOGRAPHIC>",
      "position" : 7
    },
    {
      "token" : "語",
      "start_offset" : 10,
      "end_offset" : 11,
      "type" : "<IDEOGRAPHIC>",
      "position" : 8
    },
    {
      "token" : "言",
      "start_offset" : 11,
      "end_offset" : 12,
      "type" : "<IDEOGRAPHIC>",
      "position" : 9
    }
  ]
}

接下來使用我們的IK

ik 帶有兩個分詞器 
ik_max_word ：會將文字做最細粒度的拆分；儘可能多的拆分出詞語，拼接各種可能的組合 。
ik_smart：會做最粗粒度的拆分；已被分出的詞語將不會再次被其它詞語佔有 。

curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=ik_smart&pretty" -d 'PHP是世界上最好的語言'  //ik_smart方式
{
  "tokens" : [
    {
      "token" : "php",
      "start_offset" : 0,
      "end_offset" : 3,
      "type" : "ENGLISH",
      "position" : 0
    },
    {
      "token" : "世界上",
      "start_offset" : 4,
      "end_offset" : 7,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "最好",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "CN_WORD",
      "position" : 2
    },
    {
      "token" : "語言",
      "start_offset" : 10,
      "end_offset" : 12,
      "type" : "CN_WORD",
      "position" : 3
    }
  ]
}
curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=ik_max_word&pretty" -d 'PHP是世界上最好的語言'    //ik_max_word方式
{
  "tokens" : [
    {
      "token" : "php",
      "start_offset" : 0,
      "end_offset" : 3,
      "type" : "ENGLISH",
      "position" : 0
    },
    {
      "token" : "世界上",
      "start_offset" : 4,
      "end_offset" : 7,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "世界",
      "start_offset" : 4,
      "end_offset" : 6,
      "type" : "CN_WORD",
      "position" : 2
    },
    {
      "token" : "上",
      "start_offset" : 6,
      "end_offset" : 7,
      "type" : "CN_CHAR",
      "position" : 3
    },
    {
      "token" : "最好",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "CN_WORD",
      "position" : 4
    },
    {
      "token" : "語言",
      "start_offset" : 10,
      "end_offset" : 12,
      "type" : "CN_WORD",
      "position" : 5
    }
  ]
}

區別很明顯~

相關文章