pandas.DataFrame.groupby#
Group DataFrame using a mapper or by a Series of columns.
A groupby operation involves some combination of splitting the object, applying a function, and combining the results. This can be used to group large amounts of data and compute operations on these groups.
Parameters by mapping, function, label, pd.Grouper or list of such
Used to determine the groups for the groupby. If by is a function, it’s called on each value of the object’s index. If a dict or Series is passed, the Series or dict VALUES will be used to determine the groups (the Series’ values are first aligned; see .align() method). If a list or ndarray of length equal to the selected axis is passed (see the groupby user guide), the values are used as-is to determine the groups. A label or list of labels may be passed to group by the columns in self . Notice that a tuple is interpreted as a (single) key.
Split along rows (0) or columns (1). For Series this parameter is unused and defaults to 0.
level int, level name, or sequence of such, default None
If the axis is a MultiIndex (hierarchical), group by a particular level or levels. Do not specify both by and level .
as_index bool, default True
For aggregated output, return object with group labels as the index. Only relevant for DataFrame input. as_index=False is effectively “SQL-style” grouped output.
sort bool, default True
Sort group keys. Get better performance by turning this off. Note this does not influence the order of observations within each group. Groupby preserves the order of rows within each group.
Changed in version 2.0.0: Specifying sort=False with an ordered categorical grouper will no longer sort the values.
When calling apply and the by argument produces a like-indexed (i.e. a transform ) result, add group keys to index to identify pieces. By default group keys are not included when the result’s index (and column) labels match the inputs, and are included otherwise.
Changed in version 1.5.0: Warns that group_keys will no longer be ignored when the result from apply is a like-indexed Series or DataFrame. Specify group_keys explicitly to include the group keys or not.
Changed in version 2.0.0: group_keys now defaults to True .
This only applies if any of the groupers are Categoricals. If True: only show observed values for categorical groupers. If False: show all values for categorical groupers.
dropna bool, default True
If True, and if group keys contain NA values, NA values together with row/column will be dropped. If False, NA values will also be treated as the key in groups.
Returns a groupby object that contains information about the groups.
Convenience method for frequency conversion and resampling of time series.
See the user guide for more detailed usage and examples, including splitting an object into groups, iterating through groups, selecting a group, aggregation, and more.
>>> df = pd.DataFrame('Animal': ['Falcon', 'Falcon', . 'Parrot', 'Parrot'], . 'Max Speed': [380., 370., 24., 26.]>) >>> df Animal Max Speed 0 Falcon 380.0 1 Falcon 370.0 2 Parrot 24.0 3 Parrot 26.0 >>> df.groupby(['Animal']).mean() Max Speed Animal Falcon 375.0 Parrot 25.0
Hierarchical Indexes
We can groupby different levels of a hierarchical index using the level parameter:
>>> arrays = [['Falcon', 'Falcon', 'Parrot', 'Parrot'], . ['Captive', 'Wild', 'Captive', 'Wild']] >>> index = pd.MultiIndex.from_arrays(arrays, names=('Animal', 'Type')) >>> df = pd.DataFrame('Max Speed': [390., 350., 30., 20.]>, . index=index) >>> df Max Speed Animal Type Falcon Captive 390.0 Wild 350.0 Parrot Captive 30.0 Wild 20.0 >>> df.groupby(level=0).mean() Max Speed Animal Falcon 370.0 Parrot 25.0 >>> df.groupby(level="Type").mean() Max Speed Type Captive 210.0 Wild 185.0
We can also choose to include NA in group keys or not by setting dropna parameter, the default setting is True .
>>> l = [[1, 2, 3], [1, None, 4], [2, 1, 3], [1, 2, 2]] >>> df = pd.DataFrame(l, columns=["a", "b", "c"])
>>> df.groupby(by=["b"]).sum() a c b 1.0 2 3 2.0 2 5
>>> df.groupby(by=["b"], dropna=False).sum() a c b 1.0 2 3 2.0 2 5 NaN 1 4
>>> l = [["a", 12, 12], [None, 12.3, 33.], ["b", 12.3, 123], ["a", 1, 1]] >>> df = pd.DataFrame(l, columns=["a", "b", "c"])
>>> df.groupby(by="a").sum() b c a a 13.0 13.0 b 12.3 123.0
>>> df.groupby(by="a", dropna=False).sum() b c a a 13.0 13.0 b 12.3 123.0 NaN 12.3 33.0
When using .apply() , use group_keys to include or exclude the group keys. The group_keys argument defaults to True (include).
>>> df = pd.DataFrame('Animal': ['Falcon', 'Falcon', . 'Parrot', 'Parrot'], . 'Max Speed': [380., 370., 24., 26.]>) >>> df.groupby("Animal", group_keys=True).apply(lambda x: x) Animal Max Speed Animal Falcon 0 Falcon 380.0 1 Falcon 370.0 Parrot 2 Parrot 24.0 3 Parrot 26.0
>>> df.groupby("Animal", group_keys=False).apply(lambda x: x) Animal Max Speed 0 Falcon 380.0 1 Falcon 370.0 2 Parrot 24.0 3 Parrot 26.0
GroupBy#
GroupBy objects are returned by groupby calls: pandas.DataFrame.groupby() , pandas.Series.groupby() , etc.
Indexing, iteration#
Construct DataFrame from group with provided name.
Construct DataFrame from group with provided name.
A Grouper allows the user to specify a groupby instruction for an object.
Function application helper#
Helper for column specific aggregation with control over output column names.
Function application#
Apply function func group-wise and combine the results together.
Apply function func group-wise and combine the results together.
Aggregate using one or more operations over the specified axis.
Aggregate using one or more operations over the specified axis.
Aggregate using one or more operations over the specified axis.
Aggregate using one or more operations over the specified axis.
Call function producing a same-indexed Series on each group.
Call function producing a same-indexed DataFrame on each group.
Apply a func with arguments to this GroupBy object and return its result.
Apply a func with arguments to this GroupBy object and return its result.
Filter elements from groups that don’t satisfy a criterion.
Filter elements from groups that don’t satisfy a criterion.
DataFrameGroupBy computations / descriptive stats#
Return True if all values in the group are truthful, else False.
Return True if any value in the group is truthful, else False.
Compute pairwise correlation of columns, excluding NA/null values.
Compute pairwise correlation.
Compute count of group, excluding missing values.
Compute pairwise covariance of columns, excluding NA/null values.
Number each item in each group from 0 to the length of that group — 1.
Cumulative max for each group.
Cumulative min for each group.
Cumulative product for each group.
Cumulative sum for each group.
Generate descriptive statistics.
First discrete difference of element.
Fill NA/NaN values using the specified method within groups.
Compute the first non-null entry of each column.
Return first n rows of each group.
Return index of first occurrence of maximum over requested axis.
Return index of first occurrence of minimum over requested axis.
Compute the last non-null entry of each column.
Compute max of group values.
Compute mean of groups, excluding missing values.
Compute median of groups, excluding missing values.
Compute min of group values.
Number each group from 0 to the number of groups — 1.
Take the nth row from each group if n is an int, otherwise a subset of rows.
Return DataFrame with counts of unique elements in each position.
Compute open, high, low and close values of a group, excluding missing values.
Calculate pct_change of each value to previous entry in group.
Compute prod of group values.
Return group values at the given quantile, a la numpy.percentile.
Provide the rank of values within each group.
Provide resampling when using a TimeGrouper.
Return a rolling grouper, providing rolling functionality per group.
Return a random sample of items from each group.
Compute standard error of the mean of groups, excluding missing values.
Shift each group by periods observations.
Return unbiased skew within groups.
Compute standard deviation of groups, excluding missing values.
Compute sum of group values.
Compute variance of groups, excluding missing values.
Return last n rows of each group.
Return the elements in the given positional indices in each group.
Return a Series or DataFrame containing counts of unique rows.
SeriesGroupBy computations / descriptive stats#
Return True if all values in the group are truthful, else False.
Return True if any value in the group is truthful, else False.
Compute correlation with other Series, excluding missing values.
Compute count of group, excluding missing values.
Compute covariance with Series, excluding missing values.
Number each item in each group from 0 to the length of that group — 1.
Cumulative max for each group.
Cumulative min for each group.
Cumulative product for each group.
Cumulative sum for each group.
Generate descriptive statistics.
First discrete difference of element.
Fill NA/NaN values using the specified method within groups.
Compute the first non-null entry of each column.
Return first n rows of each group.
Compute the last non-null entry of each column.
Return the row label of the maximum value.
Return the row label of the minimum value.
Return boolean if values in the object are monotonically increasing.
Return boolean if values in the object are monotonically decreasing.
Compute max of group values.
Compute mean of groups, excluding missing values.
Compute median of groups, excluding missing values.
Compute min of group values.
Number each group from 0 to the number of groups — 1.
Return the largest n elements.
Return the smallest n elements.
Take the nth row from each group if n is an int, otherwise a subset of rows.
Return number of unique elements in the group.
Return unique values of Series object.
Compute open, high, low and close values of a group, excluding missing values.
Calculate pct_change of each value to previous entry in group.
Compute prod of group values.
Return group values at the given quantile, a la numpy.percentile.
Provide the rank of values within each group.
Provide resampling when using a TimeGrouper.
Return a rolling grouper, providing rolling functionality per group.
Return a random sample of items from each group.
Compute standard error of the mean of groups, excluding missing values.
Shift each group by periods observations.
Return unbiased skew within groups.
Compute standard deviation of groups, excluding missing values.
Compute sum of group values.
Compute variance of groups, excluding missing values.
Return last n rows of each group.
Return the elements in the given positional indices in each group.
Plotting and visualization#
Make box plots from DataFrameGroupBy data.
Make a histogram of the DataFrame’s columns.
Draw histogram of the input series using matplotlib.
Make plots of Series or DataFrame.
Make plots of Series or DataFrame.