用python實現的可以自動補全的字首樹

hh_good_dog發表於2016-12-23

1,以下是程式碼部分

import os,sys
import json
class TrieTree:
    def __init__(self,is_debug=1,is_sentence=0):
        self.tree = None
        self.tree = {}
        self.is_debug = is_debug
        self.is_sentence = is_sentence
        self.prefix_list = []
    def addFromFile(self,filePath):
        with open(filePath) as f:
            for line in f:
                line_list = line.strip().strip("#").split("#")
                main_word = line_list[0].strip().split()
                if not self.is_sentence:
                    sub_word_list = [
                        u.replace(" ","") for u in line_list
                    ]
                else:
                    sub_word_list = line_list

                for i,w in enumerate(main_word):
                    if i == 0:
                        target_dict = self.tree
                    else:
                        target_dict = target_dict[main_word[i-1]]
                    if w not in target_dict:
                        target_dict[w] = {}
                        target_dict[w]["##cnt"] = 1
                        target_dict[w]["##terminal"] = []
                        target_dict[w]["##wordTag"] = 0
                    else:
                        target_dict[w]["##cnt"] += 1
                    if i== len(main_word)-1:
                        target_dict[w]["##terminal"].extend(sub_word_list)
                        target_dict[w]["##wordTag"] = 1
        if self.is_debug:
            context = json.dumps(self.tree,indent=2,ensure_ascii=False)
            print>>file("./debug.json","w"),context
    def searchPrefix(self,prefix_string):
        self.prefix_list = []
        target_dict = self.tree
        if not self.tree:
            return self.prefix_list
        if self.is_sentence:
            prefix_string = prefix_string.strip().split(" ")
        for i,w in enumerate(prefix_string):
            if w not in target_dict:
                return self.prefix_list
            else: 
                target_dict = target_dict[w]
        def deepSearch(target_dict):
            if len(target_dict.keys())==3:
                self.prefix_list.extend(target_dict["##terminal"])
                return
            else:
                self.prefix_list.extend(target_dict["##terminal"])
                for k in target_dict.keys():
                    if k not in ["##terminal","##cnt","##wordTag"]:
                        deepSearch(target_dict[k])
        deepSearch(target_dict)
        return self.prefix_list



if __name__ == "__main__":
    trie = TrieTree(is_debug=1,is_sentence=1)
    trie.addFromFile(sys.argv[1])
    while 1:
        raw=raw_input("Please input:")
        print trie.searchPrefix(raw)

2,以下是測試用例部分,將下面的英文句子貼上到一個檔名字是sent.d中;

Hi, my name is Steve.#
It’s nice to meet you.#
It’s a pleasure to meet you I’m Jack.#
What do you do for a living.#
I work at a restaurant.#
I work at a bank.#
I work in a software company.#
I’m a dentist.#
What is your name.#
What was that again.#
Excuse me.#
Pardon me.#
Are you ready?#
Are you free now?#
Are you Mr. Murthy?#
Are you angry with me?#
Are you afraid of them?#
Are you tired?#
Are you married?#
Are you employed?#
Are you interested in that?#
Are you awake?#
Are you aware of that?#
Are you a relative of Mr. Mohan?#
Are you not well?#
Are they your relatives?#
Are they from abroad?#
Are the shops open?#
Are you satisfied now?#
Are you joking?#

3,測試過程
在linux shell中執行:
python trieTree.py sent.d
即可輸入一個完整的單詞字首進行查詢了!

** 這裡你可能會有疑問,這個演算法只能是按照字首搜尋,即
按照2裡面的例子來看,輸入Are,只能得到一Are 開頭的句子,輸入Are you 只能得到以Are you 開頭的句子,如果我想知道 所有含有單詞shops的句子呢?該如何處理,這個時候 “字尾樹”就會發揮作用了,名字為字尾樹,實則不然,其實是把所有句子的字尾單元都壓入到一個字首樹中,例如
Are you a lucky dog?
這個句子的所有的字尾就是
Are you a lucky dog?
you lucky dog?
lucky dog?
dog?
把每個句子的所有的字尾都壓入到字首樹中,那麼是不是就會很方便的查詢到含有某個單詞的所有句子了呢?

相關文章