如何有效地将pos_tag_sents（）应用于熊猫数据框

如何有效地将pos_tag_sents（）应用于熊猫数据框

输入值

$ cat test.csv ID,Task,label,Text1,Collect Information,no response,cozily married practical athletics Mr. Brown flat2,New Credit,no response,active married expensive soccer Mr. Chang flat3,Collect Information,response,healthy single expensive badminton Mrs. Green flat4,Collect Information,response,cozily married practical soccer Mr. Brown hierachical5,Collect Information,response,cozily single practical badminton Mr. Brown flat

TL; DR

>>> from nltk import word_tokenize, pos_tag, pos_tag_sents>>> import pandas as pd>>> df = pd.read_csv('test.csv', sep=',')>>> df['Text']0    cozily married practical athletics Mr. Brown flat1       active married expensive soccer Mr. Chang flat2    healthy single expensive badminton Mrs. Green ...3    cozily married practical soccer Mr. Brown hier...4     cozily single practical badminton Mr. Brown flatName: Text, dtype: object>>> texts = df['Text'].tolist()>>> tagged_texts = pos_tag_sents(map(word_tokenize, texts))>>> tagged_texts[[('cozily', 'RB'), ('married', 'JJ'), ('practical', 'JJ'), ('athletics', 'NNS'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('flat', 'JJ')], [('active', 'JJ'), ('married', 'VBD'), ('expensive', 'JJ'), ('soccer', 'NN'), ('Mr.', 'NNP'), ('Chang', 'NNP'), ('flat', 'JJ')], [('healthy', 'JJ'), ('single', 'JJ'), ('expensive', 'JJ'), ('badminton', 'NN'), ('Mrs.', 'NNP'), ('Green', 'NNP'), ('flat', 'JJ')], [('cozily', 'RB'), ('married', 'JJ'), ('practical', 'JJ'), ('soccer', 'NN'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('hierachical', 'JJ')], [('cozily', 'RB'), ('single', 'JJ'), ('practical', 'JJ'), ('badminton', 'NN'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('flat', 'JJ')]]>>> df['POS'] = tagged_texts>>> df   ID      Task        label     1  Collect Information  no response   1   2New Credit  no response   2   3  Collect Information     response   3   4  Collect Information     response   4   5  Collect Information     response    Text    cozily married practical athletics Mr. Brown flat   1     active married expensive soccer Mr. Chang flat   2  healthy single expensive badminton Mrs. Green ...   3  cozily married practical soccer Mr. Brown hier...   4   cozily single practical badminton Mr. Brown flat     POS  0  [(cozily, RB), (married, JJ), (practical, JJ),...  1  [(active, JJ), (married, VBD), (expensive, JJ)...  2  [(healthy, JJ), (single, JJ), (expensive, JJ),...  3  [(cozily, RB), (married, JJ), (practical, JJ),...  4  [(cozily, RB), (single, JJ), (practical, JJ), ...

---

在长：

首先，您可以将

Text

列提取到字符串列表中：

texts = df['Text'].tolist()

然后可以应用该

word_tokenize

功能：

map(word_tokenize, texts)

注意，@Boud的建议几乎是相同的，使用

df.apply

：

df['Text'].apply(word_tokenize)

然后将标记化的文本转储到字符串列表中：

df['Text'].apply(word_tokenize).tolist()

然后，您可以使用

pos_tag_sents

：

pos_tag_sents( df['Text'].apply(word_tokenize).tolist() )

然后将列添加回Dataframe中：

df['POS'] = pos_tag_sents( df['Text'].apply(word_tokenize).tolist() )