因此,这是我在@Not_a_Golfer和@nramirezuy的大力帮助下解决问题的方法,我只是在使用它们的两个建议:
class NoFollowSpider(CrawlSpider):def __init__(self, crawl_pages=False, moreparams=None, *args, **kwargs): super(NoFollowSpider, self).__init__(*args, **kwargs) # Set the class member from here if (crawl_pages is True): NoFollowSpider.rules = ( Rule (SgmllinkExtractor(allow=("", ),), callback="parse_items", follow= True),) # Then recompile the Rules super(NoFollowSpider, self)._compile_rules() # Keep going as before self.moreparams = moreparams
谢谢大家的帮助!
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)