python – 在Windows服务器上使用scrapy进行线程阻塞

概述我在 Windows服务器上运行以下代码时出错 scrapy shell "http://www.yahoo.com" 但我对没有重定向到https的网站没有问题 我认为问题在于线程阻塞.请有人帮帮我 这是错误消息 C:\Documents and Settings\mahyar>scrapy shell "http://www.yahoo.com"2014-03-03 15:49:38-060 我在 Windows服务器上运行以下代码时出错

scrapy shell "http://www.yahoo.com"

但我对没有重定向到https的网站没有问题
我认为问题在于线程阻塞.请有人帮帮我

这是错误消息

C:\documents and Settings\mahyar>scrapy shell "http://www.yahoo.com"2014-03-03 15:49:38-0600 [scrapy] INFO: Scrapy 0.22.2 started (bot: scrapybot)2014-03-03 15:49:38-0600 [scrapy] INFO: Optional features available: ssl,http112014-03-03 15:49:38-0600 [scrapy] INFO: OverrIDden settings: {'LOGSTATS_INTERVAL': 0}2014-03-03 15:49:38-0600 [scrapy] INFO: Enabled extensions: TelnetConsole,CloseSpIDer,WebService,CoreStats,SpIDerState2014-03-03 15:49:38-0600 [scrapy] INFO: Enabled downloader mIDdlewares: httpAuthMIDdleware,DownloadTimeoutMIDdleware,UserAgentMIDdleware,RetryMIDdleware,DefaultheadersMIDdleware,MetaRefreshMIDdleware,httpCompressionMIDdleware,RedirectMIDdleware,cookiesMIDdleware,ChunkedTransferMIDdleware,DownloaderStats2014-03-03 15:49:38-0600 [scrapy] INFO: Enabled spIDer mIDdlewares: httpErrorMIDdleware,OffsiteMIDdleware,RefererMIDdleware,UrlLengthMIDdleware,DepthMIDdleware2014-03-03 15:49:38-0600 [scrapy] INFO: Enabled item pipelines:2014-03-03 15:49:38-0600 [scrapy] DEBUG: Telnet console Listening on 0.0.0.0:60232014-03-03 15:49:38-0600 [scrapy] DEBUG: Web service Listening on 0.0.0.0:60802014-03-03 15:49:38-0600 [default] INFO: SpIDer opened2014-03-03 15:49:38-0600 [default] DEBUG: Redirecting (301) to <GET https://www.yahoo.com/> from <GET http://www.yahoo.com>Traceback (most recent call last):  file "c:\Python27\lib\runpy.py",line 162,in _run_module_as_main    "__main__",fname,loader,pkg_name)  file "c:\Python27\lib\runpy.py",line 72,in _run_code    exec code in run_globals  file "c:\Python27\lib\site-packages\scrapy\cmdline.py",line 168,in <module>    execute()  file "c:\Python27\lib\site-packages\scrapy\cmdline.py",line 143,in execute    _run_print_help(parser,_run_command,cmd,args,opts)  file "c:\Python27\lib\site-packages\scrapy\cmdline.py",line 89,in _run_print_help    func(*a,**kw)  file "c:\Python27\lib\site-packages\scrapy\cmdline.py",line 150,in _run_command    cmd.run(args,opts)  file "c:\Python27\lib\site-packages\scrapy\commands\shell.py",line 50,in run    shell.start(url=url,spIDer=spIDer)  file "c:\Python27\lib\site-packages\scrapy\shell.py",line 45,in start    self.fetch(url,spIDer)  file "c:\Python27\lib\site-packages\scrapy\shell.py",line 90,in fetch    reactor,self._schedule,request,spIDer)  file "c:\Python27\lib\site-packages\twisted\internet\threads.py",line 122,in blockingCallFromThread    result.raiseException()  file "<string>",line 2,in raiseExceptionOverflowError: integer 2147486719 does not fit '32-bit int'

解决方法 看起来您正在运行32位版本的windows,而Scrapy需要64位 *** 作系统. 总结

以上是内存溢出为你收集整理的python – 在Windows服务器上使用scrapy进行线程阻塞全部内容，希望文章能够帮你解决python – 在Windows服务器上使用scrapy进行线程阻塞所遇到的程序开发问题。

如果觉得内存溢出网站内容还不错，欢迎将内存溢出网站推荐给程序员好友。