python获取网页数据

概述第一:下载图片importurllib.requestresponse=urllib.request.urlopen('http://placekitten.com/g/500/600')cat_img=response.read()withopen('cat_500_600.jpg','wb')asf:f.write(cat_img)第二:使用有道翻译,加请求头,加访问dataimporturllib.

第一:下载图片

import urllib.requestresponse = urllib.request.urlopen('http://placekitten.com/g/500/600')cat_img = response.read()with open('cat_500_600.jpg','wb') as f:    f.write(cat_img)

第二:使用有道翻译,加请求头,加访问data

import urllib.requestimport urllib.parseimport Jsoncontent = input('输入需要翻译的内容:')url = 'https://fanyi.youdao.com/translate?smartresult=dict&smartresult=rule'data = {}data['i']= contentdata['from']='auto'data['to']='auto'data['smartresult']='dict'data['clIEnt']='fanyIDeskweb'data['salt']='16226799592359'data['sign']='36b4d3a6f8cec877cb56f24fb64f85bb'data['lts']='1622679959235'data['bv']='9ff8102373b1562471f4b6881a5653e9'data['DOCTYPE']='Json'data['version']= '2.1'data['keyfrom']='fanyi.web'data['action']='FY_BY_REALTlME'data = urllib.parse.urlencode(data).encode('UTF-8')#添加请求头head = {}head['User-Agent'] = 'Mozilla/5.0 (windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36'req= urllib.request.Request(url,data,head)response = urllib.request.urlopen(req)HTML = response.read().decode('UTF-8')target = Json.loads(HTML)res = target['translateResult'][0][0]['tgt']print('翻译结果:%s' %(res))

第三:如何使用代理

import urllib.requesturl = 'http://www.whatismyip.com.tw/'proxy_support = urllib.request.ProxyHandler({'http':'119.6.144.73:81'})opener = urllib.request.build_opener(proxy_support)urllib.request.install_opener(opener)response = urllib.request.urlopen(url)HTML = response.read().decode('UTF-8')print(HTML)

 

总结

以上是内存溢出为你收集整理的python获取网页数据全部内容，希望文章能够帮你解决python获取网页数据所遇到的程序开发问题。

如果觉得内存溢出网站内容还不错，欢迎将内存溢出网站推荐给程序员好友。