那就是w3lib。w3lib 是scrapy的基础插件,用来处理html,相当好用,以下是例子:
是不是很赞?另外,w3lib还提供了多种高度自由的方法来进行字符串清洗:
OK,在文章的最后分享两个demo,用于生成headers字典和cookie字典的,当然写的不太好,希望能对你有所帮助
1、正则表达式去掉html标签代码如下:/// <Header>/// 去除 HTML tag
/// </Header>
/// <param name="HTML">源</param>
/// <returns>结果</returns> public static string StripHTML(string HTML) //google "StripHTML" 得到{ string[] Regexs =
{
@"<script[^>]*?>.*?</script>",
@"<(\/\s*)?!?((\w+:)?\w+)(\w+(\s*=?\s*(([""'])(\\[""'tbnr]|[^\7])*?\7|\w+)|.{0})|\s)*?(\/\s*)?>",
@"([\r\n])[\s]+",
@"&(quot|#34)",
@"&(amp|#38)",
@"&(lt|#60)",
@"&(gt|#62)",
@"&(nbsp|#160)",
@"&(iexcl|#161)",
@"&(cent|#162)",
@"&(pound|#163)",
@"&(copy|#169)",
@"(\d+)",
@"-->",
@"<!--.*\n"
}
string[] Replaces =
{
"",
"",
"",
"\"",
"&",
"<",
">",
" ",
"\xa1", //chr(161),"\xa2", //chr(162),"\xa3", //chr(163),"\xa9", //chr(169),"",
"\r\n",
""
}
string s = HTML
for (int i = 0i <Regexs.Lengthi++)
{
s = new Regex(Regexs[i], RegexOptions.Multiline | RegexOptions.IgnoreCase).Replace(s, Replaces[i])
}
s.Replace("<", "")
s.Replace(">", "")
s.Replace("\r\n", "")
return s
}
}
2、可以直接复制到txt,然后保存成为.html,在浏览器中设置即可!
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)