Python-爬取CVE漏洞庫?

Python-爬取CVE漏洞庫?

最近吧準備復現一下近幾年的漏洞?，一個一個的去找太麻煩了。今天做到第幾頁後面過幾天再來可能就不記得了。所以我想這搞個爬蟲給他爬下來做個excel表格，那就清楚多了。?奈何還沒寫過爬蟲，之前就一直對爬蟲挺感興趣的，但是一直沒去研究過。今天正好碰到了，躲是躲不掉了，我也嘗試找找網上有沒有現成的，畢竟我們強大的網際網路，找是找到了，還找到好幾個，奈何都用不了?，並且還看不太懂大佬寫的程式碼。今天搞了一下午搞出了個蹩腳的程式碼?，以後慢慢的再去改進改進吧，畢竟我覺著速度不太快。爬了36000多條資料就花了我20分鐘?，後面慢慢改進吧。有大佬指導指導更好！?如果程式碼有什麼問題可以直接在下面評論！，下面上程式碼：

python3寫的程式碼：

#作者：胖三斤的部落格
#時間：2021/11/5
import requests
from bs4 import BeautifulSoup
import xlsxwriter

workbook = xlsxwriter.Workbook('loudong.xlsx') # 建立檔案

worksheet = workbook.add_worksheet()
worksheet.write(0,0,'URL')      #這個是寫進第一行第一列
worksheet.write(0,1,'cve')      #這個是寫進第一行第二列
worksheet.write(0,2,'time')     #後面以此類推
worksheet.write(0,3,'name')

k=1
i=1
for j in range(1,3601):       # 開始頁數到結束頁數，自行設定
    burp0_url = f"http://cve.scap.org.cn:80/vulns/{j}?view=global"
    burp0_cookies = {"_csrf_token": "629b8310c3efb5aca85b39726ef56d29b505dc91", "session": "eyJfY3NyZl90b2tlbiI6IjYyOWI4MzEwYzNlZmI1YWNhODViMzk3MjZlZjU2ZDI5YjUwNWRjOTEiLCJfZnJlc2giOmZhbHNlfQ.YYTTIA.E-cwfm_arSSLqc772cS3GqCIPu0", "Hm_lvt_1ac51b9b492db88525810a29c7aa73cd": "1636092045", "Hm_lpvt_1ac51b9b492db88525810a29c7aa73cd": "1636094752"}
    burp0_headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8", "Accept-Language": "zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2", "Accept-Encoding": "gzip, deflate", "Connection": "close", "Referer": "http://cve.scap.org.cn/vulns/2?view=global", "Upgrade-Insecure-Requests": "1"}
    data = requests.get(burp0_url, headers=burp0_headers, cookies=burp0_cookies).text

    soup = BeautifulSoup(data, 'lxml')

    for link in soup.find_all('td'):
        if i%6 == 1:
            href = link.a.attrs['href']
            cve = link.a.string.strip()
            worksheet.write(k, 0, "http://cve.scap.org.cn"+href)
            worksheet.write(k, 1, cve)
        if i%6 == 2:
            time = link.string
            worksheet.write(k, 2, time)
        if i%6 == 4:
            name = link.string
            worksheet.write(k, 3, name)
        if i%6 == 0:
            k = k+1
        i = i+1
    print(f"已爬取資料：第{j}條")
workbook.close()

---