使用curl斷點續傳下載檔案

天琊藍發表於2021-07-11

辦公網路網速不是很好,使用Chrome下載一些軟體時不時會中斷,噁心的是Chrome居然不支援斷點續傳下載(為什麼chrome的下載不支援斷點續傳呢?),迅雷自然是不能裝的,那怎麼辦?還好我有大名鼎鼎的curl,看官網的介紹,是不是有點6到沒朋友?~~

Supports...

DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, HTTP/3, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, SCRAM-SHA, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.

What's curl used for?

curl is used in command lines or scripts to transfer data. curl is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, settop boxes, media players and is the Internet transfer engine for thousands of software applications in over ten billion installations.

curl is used daily by virtually every Internet-using human on the globe.

而且 Windows 10 已經內建支援curl(Tar and Curl Come to Windows), 通過 PowerShell/CMD 可以直接使用,一口氣上六樓,很方便?!

C:\>curl
curl: try 'curl --help' for more information

C:\>curl -V -v
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: 2017-11-14, security patched: 2019-11-05
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL

言歸正傳,下面我們使用酷酷的`curl`下載一些檔案(注:以下演示均在高速網路⚡下進行)
  1. 最簡單的使用curl --remote-name(-O) URL 或者 curl --output(-o) fileName URL, 以下載最新64位windows版本Curl為例

    C:\>mkdir Software
    
    C:\>cd Software
    
    C:\Software>curl -O https://curl.se/windows/dl-7.77.0_2/curl-7.77.0_2-win64-mingw.zip
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
     100 5154k  100 5154k    0     0  5154k      0  0:00:01 --:--:--  0:00:01 6467k
    

  1. 使用-continue-at(-C)斷點續傳下載,這次用PowerShell下載一個大檔案(SSMS),關於該選項的使用,參見以下官方使用者手冊註解

    -C, --continue-at

    Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped, counting from the beginning of the source file before it is transferred to the destination. If used with uploads, the FTP server command SIZE will not be used by curl.

    Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

    If this option is used several times, the last one will be used.

    • 在Powershell中使用需要帶上字尾名,即curl.exe,不然不能正確識別

      PS C:\Software> curl -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      Invoke-WebRequest : 無法處理引數,因為引數名稱“C”具有二義性。可能的匹配項包括:  -Credential -CertificateThumbprint -Certificate -ContentType。
      所在位置 行:1 字元: 6
      + curl -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f ...
      +      ~~
          + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest],ParameterBindingException
          + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
      
      PS C:\Software> curl --continue-at - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      Invoke-WebRequest : 找不到接受實際引數“-”的位置形式引數。
      所在位置 行:1 字元: 1
      + curl --continue-at - -O https://download.microsoft.com/download/4/6/8 ...
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest],ParameterBindingException
          + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
      
    • 通過關閉無線/有線網路,模擬網路不穩定的情況

      PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
       16  565M   16 94.7M    0     0  4852k      0  0:01:59  0:00:20  0:01:39     0
      curl: (56) Send failure: Connection was reset
      
      PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      ** Resuming transfer from byte position 172521856
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
        9  470M    9 46.9M    0     0  1502k      0  0:05:20  0:00:32  0:04:48 3720k
      curl: (56) Send failure: Connection was reset
      
    • 可以使用--retry選項讓下載失敗後自動進行(此時網路沒連線,在觀察到重試現象後,開啟網路連線)

      PS C:\Software> curl.exe -C - -O --retry 10 https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      ** Resuming transfer from byte position 221771295
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 1 seconds. 10 retries left.
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 2 seconds. 9 retries left.
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 4 seconds. 8 retries left.
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 8 seconds. 7 retries left.
        0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 16 seconds. 6 retries left.
      100  423M  100  423M    0     0  4667k      0  0:01:33  0:01:33 --:--:-- 7028k
      PS C:\Software>
      
    • 下載完成後使用斷點續傳及重試選項,不會覆蓋已下載的檔案;反之,則不成立

      PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      ** Resuming transfer from byte position 666228072
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
        0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
      curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
      
      PS C:\Software> curl.exe -C - -O --retry 3 https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
      ** Resuming transfer from byte position 666228072
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
        0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
      Warning: Transient problem: FTP error Will retry in 1 seconds. 3 retries left.
        0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
      Warning: Transient problem: FTP error Will retry in 2 seconds. 2 retries left.
        0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
      Warning: Transient problem: FTP error Will retry in 4 seconds. 1 retries left.
        0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
      curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
      
      PS C:\Software> curl.exe -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      100  635M  100  635M    0     0  16.2M      0  0:00:39  0:00:39 --:--:-- 24.2M
      

對於有些資源,網速不好的時候,甚至跳轉到具體下載連結都很困難,以下載最新的 windows 64 位版本的 PostgreSQL 13.X為例,官網找到下載資源列表,頁面跳轉後一直不開始下載, 點選下載連結也沒反應。。。

別捉急,curl一樣可以幫到你。滑鼠右鍵複製下載連結,使用--head(-I) 或者 --include(-i)選項(也可使用--verbose(-v)選項,這樣,你可以看到所謂的TCP三次握手是怎樣發生的?),具體下載連結在Location首部欄位裡,然後使用上面提到的斷點續傳,DONE✌!

C:\>curl --head https://sbp.enterprisedb.com/getfile.jsp?fileid=1257713
HTTP/1.1 302 Found
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: nginx/1.16.1
Date: Sat, 10 Jul 2021 08:50:09 GMT
Location: https://get.enterprisedb.com/postgresql/postgresql-13.3-2-windows-x64.exe
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-Cache: Miss from cloudfront
Via: 1.1 a43db2746d5ea9543e11897b6654f9b6.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: HKG62-C1
X-Amz-Cf-Id: KuxMKejZWH-BMtFbzdKMCXsJE1ctLNMoSIcSYHLIkWQmbFnbexDsNg==

寫在最後,準備這篇隨筆的時候,翻閱了不少資料,發現有兩個資源是很不錯的, 一個是官網的教程, 另一個是官網提到的Everything curlEverything curl另有中文翻譯版, 找到下載章節,一個感覺就是:我有必要寫這篇部落格嗎?寫個寂寞?。。。最後,附上原文地址, 原創不易,且行且珍惜?~~

相關文章