python專案例項：抓取網頁時間段內的請求次數、頻寬

python專案例項：抓取網頁時間段內的請求次數、頻寬

需求說明：
介面地址：bandwidth_api = “https://api.upyun.com/flow/common_data”

head認證資訊：headers = {“Authorization”: “Bearer b3ea80e2-3d72-4822-ae94-b43ce05eff10”}

介面接受GET方式請求
介面接受兩個引數：start_time、end_time

如：
‘start_time’: ‘2020-12-15 00:00:00’,
‘end_time’: ‘2020-12-22 00:00:00’

需求：

1、獲取最近7天的資料，並將資料中的 time bandwidth reqs 三個資料寫入
excel表格和csv表格中 格式如：
時間 頻寬 請求數
2020/12/15 0:05 403122 161.33
2020/12/15 0:10 375068 152.88
2020/12/15 0:15 370334 157.58
2020/12/15 0:20 352434 151.63
2020/12/15 0:25 345723 147.14
2020/12/15 0:30 352961 147.07
2020/12/15 0:35 330175 140.51
2020/12/15 0:40 336152 139.36
2020/12/15 0:45 336537 141.81
2020/12/15 0:50 311625 132.85
2、並畫出折線圖

資料寫入csv表格參考：https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html

資料寫入excel表格參考：https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_excel.html

畫圖參考：https://gallery.pyecharts.org/#/Line/basic_line_chart

程式碼展示：可能token已過期，需要重新生成

知識點：requests、arrow、json等等第三方庫，函式，閉包，字典等等

# coding:utf-8
import requests
import arrow
import json
import pandas as pd
import pyecharts.options as opts
from pyecharts.charts import Line

def get_data():
    print("正在執行get_data函式") #這裡體現出閉包的好處，不管下面呼叫多少次，get_data()只呼叫一次

    st = arrow.now().shift(weeks=-1).format("YYY-MM-DD")
    et = arrow.now().format("YYY-MM-DD")

    def f2():
        bandwidth_api = "https://api.upyun.com/flow/common_data"
        headers = {"Authorization": "1ea9a9d4-03e2-43ae-9df6-c81ad1f0d2ed"}
        payload = {'start_time': st, 'end_time': et}
        r = requests.get(bandwidth_api, params=payload, headers=headers)

        list_reqs = []
        list_time = []
        list_bandwidth = []

        fd = r.text
        r = json.loads(fd)
        for data in r:
            list_reqs.append(data["reqs"])
            list_bandwidth.append(data["bandwidth"])
            list_time.append(arrow.get(data['time']).to('local').format("YYYY-MM-DD HH:mm:ss"))

        return list_reqs, list_time, list_bandwidth
    return f2

datas = get_data()


def save_excel():
    list_reqs, list_time, list_bandwidth = datas()
    df1 = pd.DataFrame(list(zip(list_bandwidth, list_time, list_reqs)),
                       columns = ["時間","頻寬","請求數"])
    df1.to_excel(r"C:\Users\ASUS\Desktop\輸出資料.xlsx",index = False)

def save_csv():
    list_reqs, list_time, list_bandwidth = datas()
    df = pd.DataFrame({'時間': list_time,
                       '頻寬': list_bandwidth,
                       '請求數': list_reqs})
    df.to_csv(r"C:\Users\ASUS\Desktop\輸出資料.csv", index=False, encoding='utf-8-sig')

def save_plot():
    list_reqs, list_time, list_bandwidth = datas()
    (
        Line()
            .set_global_opts(
            tooltip_opts=opts.TooltipOpts(is_show=False),
            xaxis_opts=opts.AxisOpts(type_="category"),
            yaxis_opts=opts.AxisOpts(
                type_="value",
                axistick_opts=opts.AxisTickOpts(is_show=True),
                splitline_opts=opts.SplitLineOpts(is_show=True),
            ),
        )
            .add_xaxis(xaxis_data=list_time)
            .add_yaxis(
            series_name="",
            y_axis=list_bandwidth,
            symbol="emptyCircle",
            is_symbol_show=True,
            label_opts=opts.LabelOpts(is_show=False),
        )
            .add_yaxis(
            series_name="",
            y_axis=list_reqs,
            symbol="emptyCircle",
            is_symbol_show=True,
            label_opts=opts.LabelOpts(is_show=False),
        )
            .render(r"C:\Users\ASUS\Desktop\chart.html")
    )

save_excel()
save_csv()
save_plot()