Gse v0.40.0 釋出了, Go 高效能分詞, 增加更多常用 API

Go 語言高效分詞, 支援英文、中文、日文等

詞典用雙陣列 trie（Double-Array Trie）實現， 分詞器演算法為基於詞頻的最短路徑加動態規劃, DAG 和 HMM (Viterbi) 演算法分詞, 新增 API 基本和結巴分詞保持一致.

支援普通、搜尋引擎、全模式、精確模式和 HMM 模式多種分詞模式，支援使用者詞典、詞性標註，可執行 JSON RPC 服務。

專案地址: https://github.com/go-ego/gse

Snake River

Add

• [NEW] Add more common APIs and more examples
• [NEW] Export seg.Find() function
• [NEW] Add AddToken and AddTokenForce func
• [NEW] Update dictionary code, add LoadNoFreq and MinTokenFreq parameter configuration
• [NEW] Update examples code, add more examples and documents
• [NEW] Optimize benchmark test tools code
• [NEW] Add seg.Find() test
• [NEW] Add AddToken and AddTokenForce test code
• [NEW] Add crf.go file directory

Update

• [NEW] Add more hmm is bool judge
• [NEW] Update test dictionary and example code
• [NEW] Update segmenter test code
• [NEW] Add clear some internal api name and update godoc
• [NEW] Add go1.12.x remove go1.10.x
• [NEW] Add AddToken and AddTokenForce example code
• [NEW] Add custom dictionary example to example.go
• [NEW] Update godoc
• [NEW] Update CHANGELOG.md

Fix

• [FIX] Optimize code wait todo load model
• [FIX] Fixed some README.md benchmark
• [FIX] Fixed link break
• [FIX] Fixed goroutines benchmark code open path

See Commits for more details, after Jan 23.