Python使用request包請求網頁亂碼解決方法
使用requests請求網頁時,返回的頁面資訊有時是亂碼,如下程式碼
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36'
}
def get_all(url,key):
params = {
'keyword':key,
'enc':'utf-8'
}
response = requests.get(url=url,params=params,headers=headers)
with open('jd.html','w',encoding='utf-8') as f:
f.write(response.text)
if __name__ == '__main__':
key = input('輸入搜尋內容:')
url = 'https://search.jd.com/Search?'
get_all(url,key)
返回的部分內容;
--------------------------------------------------------------------
注:如果你對python感興趣,我這有個學習Python基地,裡面有很多學習資料,感興趣的+Q群:895817687
--------------------------------------------------------------------
<div class="p-name p-name-type-2">
<a target="_blank" title="æžåœ°ä¼ 说çŸè¢–Tæ¤ç”·å¤å£éŸ©ç‰ˆæ½®æµçŸè¢–男士åŠè¢–tæ¤åœ†é¢†ä¼‘闲修身大ç 五分袖潮牌é’å¹´å¦ç”Ÿtæ¤è¡£æœç”·è£
430黄色 M" href="//item.jd.com/51029271063.html" οnclick="searchlog(1,51029271063,8,1,'','flagsClk=1077936264')">
<em>æžåœ°ä¼ 说çŸè¢–Tæ¤<font class="skcolor_ljg">ç”·</font>å¤å£éŸ©ç‰ˆæ½®æµçŸè¢–男士åŠè¢–tæ¤åœ†é¢†ä¼‘闲修身大ç 五分袖潮牌é’å¹´å¦ç”Ÿtæ¤è¡£æœ<font class="skcolor_ljg">ç”·è£
</font> 430黄色 M</em>
<i class="promo-words" id="J_AD_51029271063"></i>
</a>
</div>
解決方法和思路過程;
程式碼;
def get_all(url,key):
params = {
'keyword':key,
'enc':'utf-8'
}
response = requests.get(url=url,params=params,headers=headers)
# 列印出所請求頁面返回的編碼方式
print(response.encoding)
# response.apparent_encoding是通過內容分析出的編碼方式,這裡是urf-8
print(response.apparent_encoding)
# 轉碼
content = response.text.encode(response.encoding).decode(response.apparent_encoding)
print(content)
with open('jd.html','w',encoding='utf-8') as f:
f.write(content)
if __name__ == '__main__':
key = input('輸入搜尋內容:')
url = 'https://search.jd.com/Search?'
get_all(url,key)
控制檯輸出(部分);
E:\anaconda\python.exe E:/練習/最後階段/0808/jd.py
輸入搜尋內容:男裝
ISO-8859-1
utf-8
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="renderer" content="webkit">
<meta http-equiv="Cache-Control" content="max-age=300" />
<link rel="dns-prefetch" href="//search.jd.com" /><link rel="dns-prefetch" href="//item.jd.com" /><link rel="dns-prefetch" href="//list.jd.com" /><link rel="dns-prefetch" href="//p.3.cn" /><link rel="dns-prefetch" href="//misc.360buyimg.com" /><link rel="dns-prefetch" href="//nfa.jd.com" /><link rel="dns-prefetch" href="//d.jd.com" /><link rel="dns-prefetch" href="//img12.360buyimg.com" /><link rel="dns-prefetch" href="//img13.360buyimg.com" /><link rel="dns-prefetch" href="//static.360buyimg.com" /><link rel="dns-prefetch" href="//csc.jd.com" /><link rel="dns-prefetch" href="//mercury.jd.com" /><link rel="dns-prefetch" href="//x.jd.com" /><link rel="dns-prefetch" href="//wl.jd.com" /><title>男裝 - 商品搜尋 - 京東</title><meta name="Keywords" content="男裝,京東男裝" /><meta name="description" content="在京東找到了260867件男裝的類似商品,其中包含了“男裝”等型別的男裝的商品。" /><script>
window.loadFa_toJson_data={query:'%E7%94%B7%E8%A3%85'};
window.jdpts={};jdpts._st=new Date().getTime();window.pageConfig={
closeJpg : 1,
compatible: false,
searchType: 0,
jdfVersion: '2.0.0',
floatnav: 1,
price_pdos_off: 0,
actName: '',
pSource: 'search_pc',
queryParam: {
c1: 0,
c2: 1342,
c3: 0,
brand: '',
price: '',
keyword: '男裝',
page: '1'
}
};
window.searchUnit={
resizeOnebox: function(g,f,j){var g=parseInt(g),i=typeof f,h=typeof j;if(!isNaN(g)){if("string"==i&&f!=""&&g>0){$("#J_oneBoxFrame_"+f).css("height",g+10);h=="function"&&j()}else{if(i=="undefined"||i=="function"){$("#virtualWareIFrame").css("height",g>0?g+10:0);i=="function"&&f()}}}},
resizeShopbox: function(e,d){var f=0;switch(e){case 1:case 2:f=145;break;case 3:f=75;break;case 4:f=80;break;default:break}f&&$("#shopboxIFrame").css("height",f).show();typeof(d)=="string"&&(new Image().src=d)},
coupon: {}};
window.QUERY_KEYWORD='男裝';
window.REAL_KEYWORD='男裝';
</script>
<link type="text/css" rel="stylesheet" href="//misc.360buyimg.com/??jdf/1.0.0/unit/ui-base/5.0.0/ui-base.css,jdf/1.0.0/unit/shortcut/5.0.0/shortcut.css,jdf/1.0.0/unit/global-header/5.0.0/global-header.css,jdf/1.0.0/unit/myjd/5.0.0/myjd.css,jdf/1.0.0/unit/nav/5.0.0/nav.css,jdf/1.0.0/unit/shoppingcart/5.0.0/shoppingcart.css,jdf/1.0.0/unit/global-footer/5.0.0/global-footer.css,jdf/1.0.0/unit/service/5.0.0/service.css,jdf/1.0.0/unit/global-header-photo/5.0.0/global-header-photo.css,jdf/1.0.0/ui/area/1.0.0/area.css" />
<link type="text/css" rel="stylesheet" href="//misc.360buyimg.com/product/search/1.0.7/css/search.css" />
<script type="text/javascript" src="//misc.360buyimg.com/??jdf/1.0.0/unit/base/5.0.0/base.js,jdf/lib/jquery-1.6.4.js,product/module/es5-shim.js"></script>
<script>
window.SEARCH = {
cid: 1349,
ui_ver: '1.0.7',
c_category: 1342,
p_category: 0,
enable_adv: 1,
enable_prom_adwords: 1,
enable_prom_flag: 1,
enable_price: 1,
enable_stock: 2,
enable_yyk: 0,
lottery_code: '',
is_correct_hash: function(e){var a=["keyword","brand_id","activity_id","coupon_batch","ecard_id"];for(var c=0,b=a.length;c<b;c++){var d=new RegExp("(^|\\?|&)"+a[c]+"=([^&]*)(\\s|&|$)");if(d.test(e)){return true}}return false},
get_real_hash: function(){var a=window.location.hash.substr(1);if(a&&$.browser.mozilla){return location.href.substr(location.href.indexOf("#")+1)}else{return a}}
};
(function(a,b){var c=b.get_real_hash();if(b.is_correct_hash(c)){a.location.href=a.location.pathname+"?"+c;return false}else{if(a.self!=a.top||$.browser.msie&&$.browser.version<=9){var f=null,e=function(){var d=$(a).width();return 1210>d?$("html").removeClass():$("html").removeClass().addClass(d>=1210&&1390>d?"resp01":"resp02"),true};e();$(a).resize(function(){clearTimeout(f),f=setTimeout(e,20)})}}})(window,SEARCH);
</script>
</head>
<body>
<!--shortcut start-->
<div id="shortcut-2014">
<div class="w">
<ul class="fl">
<li id="ttbar-home"><i class="iconfont"></i><a href="//www.jd.com/" target="_blank">京東首頁</a></li>
<li class="dorpdown" id="ttbar-mycity"></li>
</ul>
相關文章
- Servlet中request請求Get和Post方法以及亂碼解決Servlet
- request/response解決中文亂碼
- javaweb中中文亂碼解決方法總結之response和request解決方法JavaWeb
- requests.get()爬去中文網頁亂碼解決方法網頁
- 解決URL請求中的中文亂碼問題
- 解決使用mitmprox抓包可以訪問網頁,但是使用python request 呼叫該網站介面報錯問題MIT網頁Python網站
- jboss get請求中文亂碼問題的解決[zt]
- 使用request庫的get方法發起GET請求
- java中解決request中文亂碼問題Java
- jenkins 執行可執行jar包測試中,請求亂碼解決辦法JenkinsJAR
- python爬蟲爬取網頁中文亂碼問題的解決Python爬蟲網頁
- putty亂碼解決方法
- python併發執行request請求Python
- python 生成csv亂碼問題解決方法Python
- $request 請求方法 獲取 API 的當前使用者API
- Solaris上中文請求亂碼
- 解決GAT首頁亂碼Bug
- Flutter中http請求抓包解決方案FlutterHTTP
- 網頁請求(Ajax)網頁
- Laravel核心程式碼學習 -- 請求RequestLaravel
- nodejs使用request傳送http請求NodeJSHTTP
- mysql使用source 命令亂碼問題解決方法MySql
- Java Web(二) Servlet中response、request亂碼問題解決JavaWebServlet
- IIS設定404頁面返回200碼和404頁面亂碼解決方法
- secureCRT顯示亂碼解決方法Securecrt
- MYSQL亂碼問題解決方法MySql
- Sanic request 請求資料
- axios請求超時,設定重新請求的完美解決方法iOS
- urllib.request.Request物件封裝請求物件封裝
- POST請求登入網頁網頁
- Python中get、post請求詳解(HTTP請求頭、狀態碼)PythonHTTP
- 使用 request 和 cheerio 庫來傳送 HTTP 請求HTTP
- java中亂碼問題解決方法Java
- mysql與jsp亂碼解決方法MySqlJS
- 九種解決亂碼方法總結
- Laravel 是如何請求 Request 的Laravel
- 確認請求(confirm,request)
- Charles 修改請求(Request)引數