pandas.concat#
Can also add a layer of hierarchical indexing on the concatenation axis, which may be useful if the labels are the same (or overlapping) on the passed axis number.
Parameters objs a sequence or mapping of Series or DataFrame objects
If a mapping is passed, the sorted keys will be used as the keys argument, unless it is passed, in which case the values will be selected (see below). Any None objects will be dropped silently unless they are all None in which case a ValueError will be raised.
axis , default 0
The axis to concatenate along.
join , default ‘outer’
How to handle indexes on other axis (or axes).
ignore_index bool, default False
If True, do not use the index values along the concatenation axis. The resulting axis will be labeled 0, …, n — 1. This is useful if you are concatenating objects where the concatenation axis does not have meaningful indexing information. Note the index values on the other axes are still respected in the join.
keys sequence, default None
If multiple levels passed, should contain tuples. Construct hierarchical index using the passed keys as the outermost level.
levels list of sequences, default None
Specific levels (unique values) to use for constructing a MultiIndex. Otherwise they will be inferred from the keys.
names list, default None
Names for the levels in the resulting hierarchical index.
verify_integrity bool, default False
Check whether the new concatenated axis contains duplicates. This can be very expensive relative to the actual data concatenation.
sort bool, default False
Sort non-concatenation axis if it is not already aligned.
copy bool, default True
If False, do not copy data unnecessarily.
Returns object, type of objs
When concatenating all Series along the index (axis=0), a Series is returned. When objs contains at least one DataFrame , a DataFrame is returned. When concatenating along the columns (axis=1), a DataFrame is returned.
Join DataFrames using indexes.
Merge DataFrames by indexes or columns.
The keys, levels, and names arguments are all optional.
A walkthrough of how this method fits in with other tools for combining pandas objects can be found here.
It is not recommended to build DataFrames by adding single rows in a for loop. Build a list of rows and make a DataFrame in a single concat.
>>> s1 = pd.Series(['a', 'b']) >>> s2 = pd.Series(['c', 'd']) >>> pd.concat([s1, s2]) 0 a 1 b 0 c 1 d dtype: object
Clear the existing index and reset it in the result by setting the ignore_index option to True .
>>> pd.concat([s1, s2], ignore_index=True) 0 a 1 b 2 c 3 d dtype: object
Add a hierarchical index at the outermost level of the data with the keys option.
>>> pd.concat([s1, s2], keys=['s1', 's2']) s1 0 a 1 b s2 0 c 1 d dtype: object
Label the index keys you create with the names option.
>>> pd.concat([s1, s2], keys=['s1', 's2'], . names=['Series name', 'Row ID']) Series name Row ID s1 0 a 1 b s2 0 c 1 d dtype: object
Combine two DataFrame objects with identical columns.
>>> df1 = pd.DataFrame([['a', 1], ['b', 2]], . columns=['letter', 'number']) >>> df1 letter number 0 a 1 1 b 2 >>> df2 = pd.DataFrame([['c', 3], ['d', 4]], . columns=['letter', 'number']) >>> df2 letter number 0 c 3 1 d 4 >>> pd.concat([df1, df2]) letter number 0 a 1 1 b 2 0 c 3 1 d 4
Combine DataFrame objects with overlapping columns and return everything. Columns outside the intersection will be filled with NaN values.
>>> df3 = pd.DataFrame([['c', 3, 'cat'], ['d', 4, 'dog']], . columns=['letter', 'number', 'animal']) >>> df3 letter number animal 0 c 3 cat 1 d 4 dog >>> pd.concat([df1, df3], sort=False) letter number animal 0 a 1 NaN 1 b 2 NaN 0 c 3 cat 1 d 4 dog
Combine DataFrame objects with overlapping columns and return only those that are shared by passing inner to the join keyword argument.
>>> pd.concat([df1, df3], join="inner") letter number 0 a 1 1 b 2 0 c 3 1 d 4
Combine DataFrame objects horizontally along the x axis by passing in axis=1 .
>>> df4 = pd.DataFrame([['bird', 'polly'], ['monkey', 'george']], . columns=['animal', 'name']) >>> pd.concat([df1, df4], axis=1) letter number animal name 0 a 1 bird polly 1 b 2 monkey george
Prevent the result from including duplicate index values with the verify_integrity option.
>>> df5 = pd.DataFrame([1], index=['a']) >>> df5 0 a 1 >>> df6 = pd.DataFrame([2], index=['a']) >>> df6 0 a 2 >>> pd.concat([df5, df6], verify_integrity=True) Traceback (most recent call last): . ValueError: Indexes have overlapping values: ['a']
Append a single row to the end of a DataFrame object.
>>> df7 = pd.DataFrame('a': 1, 'b': 2>, index=[0]) >>> df7 a b 0 1 2 >>> new_row = pd.Series('a': 3, 'b': 4>) >>> new_row a 3 b 4 dtype: int64 >>> pd.concat([df7, new_row.to_frame().T], ignore_index=True) a b 0 1 2 1 3 4
Pandas concat() Examples
While we believe that this content benefits our community, we have not yet thoroughly reviewed it. If you have any suggestions for improvements, please let us know by clicking the “report an issue“ button at the bottom of the tutorial.
Pandas concat() method is used to concatenate pandas objects such as DataFrames and Series. We can pass various parameters to change the behavior of the concatenation operation.
1. Pandas concat() Syntax
concat(objs, axis=0, join='outer', join_axes=None, ignore_index=False, keys=None, levels=None, names=None, verify_integrity=False, sort=None, copy=True) - objs: a sequence of pandas objects to concatenate.
- join: optional parameter to define how to handle the indexes on the other axis. The valid values are ‘inner’ and ‘outer’.
- join_axes: deprecated in version 0.25.0.
- ignore_index: if True, the indexes from the source objects will be ignored and a sequence of indexes from 0,1,2…n will be assigned to the result.
- keys: a sequence to add an identifier to the result indexes. It’s helpful in marking the source objects in the output.
- levels: a sequence to specify the unique levels to create multiindex.
- names: names for the levels in the resulting hierarchical index.
- verify_integrity: Check whether the new concatenated axis contains duplicates. It’s an expensive operation.
- sort: Sort non-concatenation axis if it is not already aligned when join is ‘outer’. Added in version 0.23.0
- copy: if False, don’t copy data unnecessarily.
Recommended Reading: Python Pandas Tutorial
2. Pandas concat() Example
Let’s look at a simple example to concatenate two DataFrame objects.
import pandas d1 = d2 = df1 = pandas.DataFrame(d1, index=) df2 = pandas.DataFrame(d2, index=) print('********\n', df1) print('********\n', df2) df3 = pandas.concat([df1, df2]) print('********\n', df3) ******** Name ID 1 Pankaj 1 2 Lisa 2 ******** Name ID 3 David 3 ******** Name ID 1 Pankaj 1 2 Lisa 2 3 David 3 Notice that the concatenation is performed row-wise i.e. 0-axis. Also, the indexes from the source DataFrame objects are preserved in the output.
3. Concatenating Along Column i.e. 1-axis
d1 = d2 = df1 = pandas.DataFrame(d1, index=) df2 = pandas.DataFrame(d2, index=) df3 = pandas.concat([df1, df2], axis=1) print('********\n', df3) ******** Name ID Role 1 Pankaj 1 Admin 2 Lisa 2 Editor The concatenation along column makes sense when the source objects contain different kinds of data of an object.
4. Assigning Keys to the Concatenated DataFrame Indexes
d1 = d2 = df1 = pandas.DataFrame(d1, index=) df2 = pandas.DataFrame(d2, index=) df3 = pandas.concat([df1, df2], keys=["DF1", "DF2"]) print('********\n', df3) ******** Name ID DF1 1 Pankaj 1 2 Lisa 2 DF2 3 David 3 5. Ignore Source DataFrame Objects in Concatenation
d1 = d2 = df1 = pandas.DataFrame(d1, index=) df2 = pandas.DataFrame(d2, index=) df3 = pandas.concat([df1, df2], ignore_index=True) print('********\n', df3) ******** Name ID 0 Pankaj 1 1 Lisa 2 2 David 3 This is useful when the indexes in the source objects don’t make much sense. So we can ignore them and assign the default indexes to the output DataFrame.
6. References
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.