python爬虫爬取各个城市历史天气及数据可视化

概述          import asyncio import aiohttp from lxml import etree import re from collections import namedtuple Args = namedtuple(‘Args‘,[‘city‘,‘year‘,‘month‘]) async def work(args): url = "http://www.ti

 

 

 

 

 

import asyncio import aiohttp from lxml import etree import re from collections import namedtuple Args = namedtuple(‘Args‘,[‘city‘,‘year‘,‘month‘]) async def work(args): url = "http://www.tianqihoubao.com/lishi/%s/month/%d%02d.HTML" % (args.city,args.year,args.month) headers = {‘User-Agent‘: ‘Mozilla/5.0 (windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/61.0.3163.100 Safari/537.36‘} async with aiohttp.ClIEntSession() as session: async with session.get(url,headers=headers,timeout=1000,verify_ssl=False) as response: HTML = await response.text() etHTML = etree.HTML(HTML) result = [re.sub("\r\n *","",item.text) for item in etHTML.xpath("//table[@class=‘b‘]/tr/td") if not re.fullmatch("\r\n *",item.text)] return result loop = asyncio.get_event_loop() tasks = [asyncio.ensure_future(work(Args(‘wuhan‘,2018,month))) for month in range(1,6)] loop.run_until_complete(asyncio.wait(tasks)) for task in tasks: print(task.result())

总结

以上是内存溢出为你收集整理的python爬虫爬取各个城市历史天气及数据可视化全部内容，希望文章能够帮你解决python爬虫爬取各个城市历史天气及数据可视化所遇到的程序开发问题。

如果觉得内存溢出网站内容还不错，欢迎将内存溢出网站推荐给程序员好友。