应用自定义groupby聚合函数在pandas python中输出二进制结果

应用自定义groupby聚合函数在pandas python中输出二进制结果

import numpy as npimport pandas as pddf = pd.Dataframe({'Buy/Sell': [1, 0, 1, 1, 0, 1, 0, 0],        'Trader': ['A', 'A', 'B', 'B', 'B', 'C', 'C', 'C']})grouped = df.groupby(['Trader'])result = grouped['Buy/Sell'].agg(['sum', 'count'])means = grouped['Buy/Sell'].mean()result['Buy/Sell'] = np.select(condlist=[means>0.5, means<0.5], choicelist=[1, 0],     default=np.nan)print(result)

产量

        Buy/Sell  sum  countTraderA NaN    1      2B   1    2      3C   0    1      3

---

我最初的答案使用了一个自定义聚合器

categorize

：

def categorize(x):    m = x.mean()    return 1 if m > 0.5 else 0 if m < 0.5 else np.nanresult = df.groupby(['Trader'])['Buy/Sell'].agg([categorize, 'sum', 'count'])result = result.rename(columns={'categorize' : 'Buy/Sell'})

虽然调用自定义函数可能很方便，但与内置聚合器（如

groupby/agg/mean

）相比，使用自定义函数时的性能通常会大大降低。内置的聚合器是Cythonized的，而自定义函数将性能降低到普通的Python
for循环速度。

当组数很大时，速度差异特别明显。例如，对于具有1000个组的10000行Dataframe，

import numpy as npimport pandas as pdnp.random.seed(2017)N = 10000df = pd.Dataframe({    'Buy/Sell': np.random.randint(2, size=N),    'Trader': np.random.randint(1000, size=N)})def using_select(df):    grouped = df.groupby(['Trader'])    result = grouped['Buy/Sell'].agg(['sum', 'count'])    means = grouped['Buy/Sell'].mean()    result['Buy/Sell'] = np.select(condlist=[means>0.5, means<0.5], choicelist=[1, 0],         default=np.nan)    return resultdef categorize(x):    m = x.mean()    return 1 if m > 0.5 else 0 if m < 0.5 else np.nandef using_custom_function(df):    result = df.groupby(['Trader'])['Buy/Sell'].agg([categorize, 'sum', 'count'])    result = result.rename(columns={'categorize' : 'Buy/Sell'})    return result

using_select

比快50倍以上

using_custom_function

：

In [69]: %timeit using_custom_function(df)10 loops, best of 3: 132 ms per loopIn [70]: %timeit using_select(df)100 loops, best of 3: 2.46 ms per loopIn [71]: 132/2.46Out[71]: 53.65853658536585