如何處理http返回型別為206的資料

killernova發表於2018-07-24

最近在用阿里雲 oss select 功能時,發現返回的資料轉成 string 的時候會有部分亂碼。但在使用 python 的 sdk 的時候就是完全正常的。由於請求頭返回的 code 是 206,所以我有點懷疑原因出在這裡,而阿里的那個文件(https://help.aliyun.com/document_detail/74054.html?spm=a2c4g.11186623.6.937.S1pqD1 我是沒太看懂。麻煩大神們給看看。

我的關鍵程式碼:

func SelectQuery(bucket, objectKey string, config *Config, s *Selector) ([]byte, error) {
    content, err := s.toXML()
    if err != nil {
        return nil, err
    }
    now := getGmtIso8601(time.Now().Unix())
    canonicalizedResource := "/" + bucket + "/" + objectKey + "?x-oss-process=csv/select"
    url := fmt.Sprintf("https://%s.oss-cn-hangzhou.aliyuncs.com/%s?x-oss-process=csv/select", bucket, objectKey)
    c := &http.Client{
        Transport: &http.Transport{
            // Proxy:           http.ProxyFromEnvironment,
            TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
            TLSNextProto:    make(map[string]func(authority string, c *tls.Conn) http.RoundTripper),
            Dial: (&net.Dialer{
                Timeout:   config.Timeout,
                KeepAlive: config.KeepAlive,
            }).Dial,
            TLSHandshakeTimeout:   config.TLSHandshakeTimeout,
            ResponseHeaderTimeout: config.ResponseHeaderTimeout,
            ExpectContinueTimeout: config.ExpectContinueTimeout,
        },
    }
    req, err := http.NewRequest("POST", url, bytes.NewReader(content))
    if err != nil {
        return nil, err
    }
    req.Header.Set("Date", now)
    req.Header.Set("Content-Length", strconv.Itoa(len(content)))
    //req.Header.Set("Content-MD5", hex.EncodeToString(auth.CalcMD5(content)))

    conn := Conn{
        Config: config,
        Client: c,
    }
    conn.SignHeader(req, canonicalizedResource)
    resp, err := c.Do(req)
    fmt.Printf("%+v\n", resp)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return body, errors.New("errors returned from ali oss")
    }
    return body, nil
}

列印資訊:

▶ go run demo.go
&{Status:206 Partial Content StatusCode:206 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Ranges:[bytes] Connection:[keep-alive] X-Oss-Storage-Class:[Standard] X-Oss-Server-Time:[60] Date:[Tue, 24 Jul 2018 05:29:38 GMT] Etag:["B9B8FE775557E10E35418410499775CF"] Server:[AliyunOSS] Content-Md5:[ubj+d1VX4Q41QYQQSZd1zw==] Last-Modified:[Mon, 23 Jul 2018 11:22:04 GMT] X-Oss-Object-Type:[Normal] X-Oss-Hash-Crc64ecma:[8407954770141203098] Content-Type:[text/csv] X-Oss-Request-Id:[5B56B942D1B3FA9711693E4D]] Body:0xc420198640 ContentLength:-1 TransferEncoding:[chunked] Close:false Uncompressed:false Trailer:map[] Request:0xc420128000 TLS:0xc4201da000}
���y�176267,0102797,30840
176269,6209470,439050
176271,2028550,111109
176273,0303758,67758
176275,0008536,227506
��176279,6980792,461257
����

在文件中,關於返回的資料,是這麼說的: > 返回結果 請求結果以一個個 Frame 形式返回。每個 Frame 的格式如下,其中 checksum 均為 CRC32: Frame-Type | Payload Length | Header Checksum | Payload | Payload Checksum <---4 bytes--><---4 bytes----------><-------4 bytes-------><variable><----4bytes----------> > 一共有三種不同的 Frame Type, 列舉如下:

更多原創文章乾貨分享,請關注公眾號
  • 如何處理http返回型別為206的資料
  • 加微信實戰群請加微信(註明:實戰群):gocnio

相關文章