小白折騰伺服器（十）：docker 下安裝 Elasticsearch+ik 分詞外掛

aen233發表於2019-05-18
伺服器DockerElasticsearch分詞

這幾天在學習電商進階的6，7章，就是使用elasticsearch重構搜尋的這兩章。
作品分享下：
http://120.27.8.8/products?order=price_desc&search=金士頓
http://120.27.8.8/products?search=記憶體條&filters=傳輸型別:DDR4|記憶體容量:8GB

學習筆記和感悟：

  1. 5.8版本下資料表設定外來鍵時主外來鍵型別必須完全一致
  2. .env中要修改，否則安裝完會報錯 “Could not parse URI”
  3. 除了資料庫裡要有資料，es中要建立索引，還要給es中新增商品資料
  4. 文件型資料庫和關係型資料庫真的不一樣，初學覺得還蠻神奇

謝謝謝謝leo老師的這麼好的教程和summer大大提供的平臺，鞠躬

以下是伺服器上使用docker搭建elasticsearch+ik的踩坑筆記
我的伺服器是阿里雲的ECS，1核2G，系統是Centos7.
nginx+php在伺服器本機搭建，mysql+redis使用docker搭建
部署專案使用deployer。

拉取映象：

// 官方映象，這步非必須，但是拉取了之後操作會快點
docker pull docker.elastic.co/elasticsearch/elasticsearch:6.5.0

給docker-compose.yml 中增加es

yml檔案真的很嚴格，空格稍微沒對齊就報錯。

version: '3'
services:
  aen233-mysql:
    image: mysql:5.7
    container_name: aen233-mysql
    environment:
      MYSQL_ROOT_PASSWORD: 123456
    ports:
    - "3306:3306"
    volumes:
    - ./mysql:/var/lib/mysql
    networks:
    - esnet

  aen233-redis:
    image: redis:alpine
    container_name: aen233-redis
    ports:
    - "6379:6379"
    volumes:
    - ./redis:/data
    networks:
    - esnet

  aen233-adminer:
    image: adminer
    container_name: aen233-adminer
    ports:
    - "8080:8080"
    links:
    - aen233-mysql:db
    networks:
    - esnet

  aen233-elasticsearch:
    build: .
    container_name: aen233-elasticsearch
    environment:
    - "discovery.type=single-node"
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    - http.cors.enabled=true
    - http.cors.allow-origin=*
    volumes:
    - ./esdata:/usr/share/elasticsearch/data
    ports:
    - "9200:9200"
    networks:
    - esnet

networks:
  esnet:

docker-compose.yml儲存後就可以敲命令了

chmod 777 esdata   //先給伺服器掛載es資料的資料夾（就是docker-compose中elasticsearch下volumes中配的資料夾）加個許可權，否則es容器會自動退出
docker-compose up -d

測試一下es，

curl http://127.0.0.1:9200/

如果返回如下就ok啦

{
  "name" : "3E1nODN",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "x-H40SveSYqvvAZycOr1bw",
  "version" : {
    "number" : "6.5.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "816e6f6",
    "build_date" : "2018-11-09T18:58:36.352602Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

這裡不是立馬就生效，大概要等10s到1min，
curl: (52) Empty reply from server
是稍等一下下就好
curl: (7) Failed to connect to 127.0.0.1 port 9200: Connection refused
是es容器exit了

docker-compose的刪除關閉是，docker-compose down
docker ps -a 檢視如果es自動退出就 docker-compose down
down了再改 docker-compose.yml , 改了 up ，容器自動退了就 down

增加ik分詞，兩條命令

docker-compose exec aen233-elasticsearch elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip

//然後要重啟es容器
docker-compose restart aen233-elasticsearch

測試一下ik， 

// 增加一個叫test001的索引
curl -X PUT http://127.0.0.1:9200/test001
// 筆記1:  返回 "acknowledged":true 才算正常
// 筆記2:  使用pretty=true引數美化返回格式，更易於閱讀
curl -X POST \
'http://127.0.0.1:9200/test001/_analyze?pretty=true' \
-H 'Content-Type: application/json' \
-d '{"text":"我們是軟體工程師","tokenizer":"ik_smart"}'

如果返回如下就ok啦

{
  "tokens" : [
    {
      "token" : "我們",
      "start_offset" : 0,
      "end_offset" : 2,
      "type" : "CN_WORD",
      "position" : 0
    },
    {
      "token" : "是",
      "start_offset" : 2,
      "end_offset" : 3,
      "type" : "CN_CHAR",
      "position" : 1
    },
    {
      "token" : "軟體",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "CN_WORD",
      "position" : 2
    },
    {
      "token" : "工程師",
      "start_offset" : 5,
      "end_offset" : 8,
      "type" : "CN_WORD",
      "position" : 3
    }
  ]
}

初學小tips：

如果你也是初學，先用最簡單的命令測試下能不能跑起來

docker run -d --name aen233-elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms512m -Xmx512m" docker.elastic.co/elasticsearch/elasticsearch:6.5.0

-d 開啟daemon模式，後臺執行，提供服務
--name 給這個容器起個名字
-p 埠對映，elasticsearch好像9200對映出來就好了
-e 配置環境變數 ES_JAVA_OPTS="-Xms512m -Xmx512m" 給它配一個可用記憶體限制，防止記憶體溢位，要不然伺服器很容易卡死

然後 docker ps -a 檢視所有的容器 ,
docker stop 容器id , docker rm 容器id
關了刪了再重來，直到成功

報錯踩坑

第一個錯：Exited (78) 55 minutes ago

查錯：docker-compose logs elasticsearch
解決：sudo sysctl -w vm.max_map_count=262144
但是這個，最好加個"ES_JAVA_OPTS=-Xms512m -Xmx512m"的環境變數，要不然很容易記憶體溢位
參考連結：github issue：Elasticsearch Container Stopped with 'Exit 78' state in Ubuntu 18.04

第二個錯：掛載本地資料 自動退出

查錯：許可權問題，需要把本地掛載的目錄改為777就可以了
解決：chmod 777 esdata
參考連結：Docker ElasticSearch掛載本地資料 報錯

第三個錯：Dockerfile裡給es配ik就報錯

結果：未解決，使用docker-compose exec + docker-compose restart 手動安裝的ik

[root@aen233 aen233-dnmp]# docker-compose up -d
Creating network "aen233-dnmp_esnet" with the default driver
Building aen233-elasticsearch
Step 1/3 : FROM docker.elastic.co/elasticsearch/elasticsearch:6.5.0
 ---> ff171d17e77c
Step 2/3 : RUN elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip
 ---> Running in c42cc0c3a221

-> Downloading https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip
[=================================================] 100%??
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Exception in thread "main" java.lang.IllegalStateException: unable to read from standard input; is standard input open and a tty attached?
    at org.elasticsearch.cli.Terminal$SystemTerminal.readText(Terminal.java:173)
    at org.elasticsearch.plugins.PluginSecurity.prompt(PluginSecurity.java:74)
    at org.elasticsearch.plugins.PluginSecurity.confirmPolicyExceptions(PluginSecurity.java:67)
    at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:801)
    at org.elasticsearch.plugins.InstallPluginCommand.install(InstallPluginCommand.java:775)
    at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:231)
    at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:216)
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
    at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
    at org.elasticsearch.cli.Command.main(Command.java:90)
    at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)
ERROR: Service 'aen233-elasticsearch' failed to build: The command '/bin/sh -c elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip' returned a non-zero code: 1
[root@aen233 aen233-dnmp]#
本作品採用《CC 協議》，轉載必須註明作者和本文連結

相關文章