1.MongoDB的安裝
MongoDB是一種非關係型資料庫
選擇你的系統對應的版本下載安裝即可
2.MongoDB配置
a.在C盤或者D盤建一個資料夾如圖mongodb
b.安裝成功后里面會有bin檔案然後再資料夾裡面新建一個data資料夾，data檔案裡面新建db資料夾
db資料夾用於儲存MongoDB資料
c.在bin檔案路徑下開啟命令列工具執行下面的命令
mongod --dbpath C:mongdbdatadb
注意：資料夾路徑以自己所建的為準
d.此時在開啟一個命令列在bin路徑下執行下面的程式碼
mongo
3.安裝第三方庫pymongo(連線MongoDB)
pip3 install pymongo4.安裝Mongodb視覺化管理工具Robomongo
安裝成功之後啟動Robomongo，在空白處點選，然後選擇Add命令，單擊Save,最後點選Connect按鈕連線到MongoDB資料庫
5.案例程式碼
引入相應的模組
import requests
from lxml import etree
import re
import pymongo
import time連線mongodb資料庫
client = pymongo.MongoClient(`localhost`, 27017)
mydb = client[`mydb`]
musictop = mydb[`musictop`]案例完整程式碼
import requests
from lxml import etree
import re
import pymongo
import time
client = pymongo.MongoClient(`localhost`, 27017)
mydb = client[`mydb`]
musictop = mydb[`musictop`]
headers = {
`User-Agent`: `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36`
}
def get_url_music(url):
html = requests.get(url, headers=headers)
selector = etree.HTML(html.text)
music_hrefs = selector.xpath(`//a[@class="nbg"]/@href`)
for music_href in music_hrefs:
get_music_info(music_href)
def get_music_info(url):
html = requests.get(url, headers=headers)
selector = etree.HTML(html.text)
name = selector.xpath(`//*[@id="wrapper"]/h1/span/text()`)[0]
author = re.findall(`表演者:.*?>(.*?)</a>`, html.text,re.S)[0]
styles = re.findall(`<span class="pl">流派:</span> (.*?)<br/>`,html.text,re.S)
if len(styles) == 0:
style = `未知`
else:
style = styles[0].strip()
time = re.findall(`發行時間:</span> (.*?)<br/>`, html.text, re.S)[0].strip()
publishers = re.findall(`出版者:.*?>(.*?)</a>`, html.text, re.S)
if len(publishers) == 0:
publishers = `未知`
else:
publishers = publishers[0].strip()
score = selector.xpath(`//*[@id="interest_sectl"]/div/div[2]/strong/text()`)[0]
print(name, author, style, time, publishers, score)
info = {
`name`: name,
`author`: author,
`style`: style,
`time`: time,
`publisher`: publishers,
`score`: score
}
musictop.insert_one(info)
if __name__ == `__main__`:
urls = [`https://music.douban.com/top250?start={}`.format(str(i)) for i in range(0, 250, 25)]
for url in urls:
get_url_music(url)
time.sleep(2)