Pandas aggregate count distinct

Pandas aggregate count distinct

Asked on December 24, 2018 in Pandas.
Add Comment


  • 2 Answer(s)

    Use the below code:

    >>> df
    date duration user_id
    0 2013-04-01 30 0001
    1 2013-04-01 15 0001
    2 2013-04-01 20 0002
    3 2013-04-02 15 0002
    4 2013-04-02 30 0002
    >>> df.groupby("date").agg({"duration": np.sum, "user_id": pd.Series.nunique})
    duration user_id
    date
    2013-04-01 65 2
    2013-04-02 45 1
    >>> df.groupby("date").agg({"duration": np.sum, "user_id": lambda x: x.nunique()})
    duration user_id
    date
    2013-04-01 65 2
    2013-04-02 45 1
    
    Answered on December 24, 2018.
    Add Comment

    The function pd.Series.nunique is much quicker:

    %time _=g.agg({"id": lambda x: x.nunique()})
    CPU times: user 3min 3s, sys: 2.94 s, total: 3min 6s
    Wall time: 3min 20s
     
    %time _=g.agg({"id": pd.Series.nunique})
    CPU times: user 3min 2s, sys: 2.44 s, total: 3min 4s
    Wall time: 3min 18s
     
    %time _=g.agg({"id": 'nunique'})
    CPU times: user 14 s, sys: 4.76 s, total: 18.8 s
    Wall time: 24.4 s
    
    Answered on December 24, 2018.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.