Python ndarray to json

Fix Python TypeError: Object of type ndarray is not JSON serializable

A NumPy array can’t be converted into a JSON string because Python doesn’t know how to handle it.

When you run the following code:

  Python responds with an error as follows:

The json library supports native Python objects and types like dict , list , tuple , str , .etc.

Fortunately, the NumPy array class has the tolist() method, which allows you to convert an array to a list.

To fix this error, call the tolist() method from your array data as shown below:

Once you convert the array to a list, the dumps() method is able to process that list to a JSON string.

Another option for solving this error is to extend the JSONEncoder class and add custom handler for NumPy arrays.

The above code defines the NumpyEncoder class, which extends the built-in JSONEncoder class and adds a default() method for handling numpy arrays.

The default() method is called by the JSONEncoder class whenever it encounters an object that it does not know how to serialize. You can override this method to provide custom serialization for specific types of objects.

1. If the object is an instance of np.ndarray , return it as a list
2. If the object is an instance of np.integer , return it as an int
3. If the object is an instance of np.floating , return it as a float

This way, you can handle different data types from NumPy using json.dumps() method.

To use the extended class, pass a cls argument when calling json.dumps() :

Extending the JSONEncoder class is also useful when you have a complex type that you want to convert to JSON.

For more information and code examples, you can visit Python json documentation.

Conclusion

The Python json module can only serialize JSON-serializable data types, such as dictionaries, lists, and primitive data types (strings, numbers, booleans).

When you serialize a NumPy array with the module, Python will respond with a TypeError: Object of type ndarray is not JSON serializable .

The solution is to convert the array to a native Python list first so that json module can process it.

Alternatively, you can extend the JSONEncoder class and let json knows how to handle complex data types you have in your project.

Take your skills to the next level ⚡️

I’m sending out an occasional email with the latest tutorials on programming, web development, and statistics. Drop your email in the box below and I’ll send new stuff straight into your inbox!

About

Hello! This website is dedicated to help you learn tech and data science skills with its step-by-step, beginner-friendly tutorials.
Learn statistics, JavaScript and other programming languages using clear examples written for people.

Search

Type the keyword below and hit enter

Источник

Python Serialize NumPy ndarray into JSON

You are here because when you try to encode/serialize NumPy array into a JSON format, you received a TypeError: Object of type ndarray is not JSON serializable . In this article, I will show you how to make NumPy array JSON serializable so that you can convert any NumPy array into JSON formatted data.

As you know The built-in json module of Python can only handle primitives types that have a direct JSON equivalent (e.g., dictionary, lists, strings, Numbers, None, etc.). To serialize the multidimensional array into JSON, We need to write a custom JSON Encoder.

Further Reading:

Custom JSON Encoder to Serialize NumPy ndarray

Python json module has a JSONEncoder class, we can extend it to get more customized output. i.e., you will have to subclass JSONEncoder so you can implement custom NumPy JSON serialization.

When we extend the JSONEncoder class, we will extend its JSON encoding scope by overriding the default() method which will be used when we execute JSONEncoder.encode(numpyArray) .

Use the cls kwarg of the json.dump() and json.dumps() method to call our custom JSON Encoder, which will convert NumPy array into JSON formatted data.

Example: json.dumps(cls=NumPyArrayEncoder)

To serialize Numpy array into JSON we need to convert it into a list structure using a tolist() function. Let’s see the demo.

Encode and Decode NumPy array to and from JSON

In this example, we try to serialize the NumPy Array into JSON String.

import json from json import JSONEncoder import numpy class NumpyArrayEncoder(JSONEncoder): def default(self, obj): if isinstance(obj, numpy.ndarray): return obj.tolist() return JSONEncoder.default(self, obj) numpyArrayOne = numpy.array([[11, 22, 33], [44, 55, 66], [77, 88, 99]]) # Serialization numpyData = encodedNumpyData = json.dumps(numpyData, cls=NumpyArrayEncoder) # use dump() to write array into file print("Printing JSON serialized NumPy array") print(encodedNumpyData) # Deserialization print("Decode JSON serialized NumPy array") decodedArrays = json.loads(encodedNumpyData) finalNumpyArray = numpy.asarray(decodedArrays["array"]) print("NumPy Array") print(finalNumpyArray)

Printing JSON serialized NumPy array Decode JSON serialized NumPy array NumPy Array [[11 22 33] [44 55 66] [77 88 99]]

Note: We used numpy.asarray() to convert data into NumPy array

Encode NumPy array into JSON and write it in a file

In most scenarios, we need to store JSON serialized NumPy array into a file so we can use it in different systems.

In this example, we will do the following:

• Convert two NumPy arrays into JSON and write it into a JSON file
• Read the JSON file, which contains JSON serialized NumPy array and convert it into actual NumPy Array.

import numpy from json import JSONEncoder import json class NumpyArrayEncoder(JSONEncoder): def default(self, obj): if isinstance(obj, numpy.ndarray): return obj.tolist() return JSONEncoder.default(self, obj) numpyArrayOne = numpy.array([[11 ,22, 33], [44, 55, 66], [77, 88, 99]]) numpyArrayTwo = numpy.array([[51, 61, 91], [121 ,118, 127]]) # Serialization numpyData = print("serialize NumPy array into JSON and write into a file") with open("numpyData.json", "w") as write_file: json.dump(numpyData, write_file, cls=NumpyArrayEncoder) print("Done writing serialized NumPy array into file") # Deserialization print("Started Reading JSON file") with open("numpyData.json", "r") as read_file: print("Converting JSON encoded data into Numpy array") decodedArray = json.load(read_file) finalNumpyArrayOne = numpy.asarray(decodedArray["arrayOne"]) print("NumPy Array One") print(finalNumpyArrayOne) finalNumpyArrayTwo = numpy.asarray(decodedArray["arrayTwo"]) print("NumPy Array Two") print(finalNumpyArrayTwo)

serialize NumPy array into JSON and write into a file Done writing serialized NumPy array into file Started Reading JSON file Converting JSON encoded data into Numpy array NumPy Array One [[11 22 33] [44 55 66] [77 88 99]] NumPy Array Two [[ 51 61 91] [121 118 127]]

Encode all NumPy types correctly into JSON

import json from json import JSONEncoder import numpy as np class NumpyArrayEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, np.integer): return int(obj) elif isinstance(obj, np.floating): return float(obj) elif isinstance(obj, np.ndarray): return obj.tolist() else: return super(NumpyArrayEncoder, self).default(obj) # Serialization numPyData = encodedNumpyData = json.dumps(numPyData, cls=NumpyArrayEncoder) # use dump() to write array into file print("Printing JSON serialized NumPy array") print(encodedNumpyData) # Deserialization print("Decode JSON serialized NumPy array") decodedArrays = json.loads(encodedNumpyData) numPyFloat = np.asarray(decodedArrays["floatSample"]) print("numPy Float") print(numPyFloat) numPyInt = np.asarray(decodedArrays["intSample"]) print("numPy Integer") print(numPyInt) numPyArange = np.asarray(decodedArrays["arangeSample"]) print("numPy arange") print(numPyArange)

Printing JSON serialized NumPy array Decode JSON serialized NumPy array numPy Float 1.2000000476837158 numPy Integer 42 numPy arange [ 0 1 2 3 4 5 6 7 8 9 10 11]

Use panads to_json method to serialize NumPy ndarray into JSON

If you use pandas for data manipulation, you can use to_json function on Series. The pandas.Series.to_json convert the object to a JSON string. we can pass NumPy array to it to get the JSON representation.

import pandas as pd import numpy numpyArray = numpy.array([[51, 61, 91], [121, 118, 127]]) df = pd.DataFrame(numpyArray, index=['row 1', 'row 2'], columns=['col 1', 'col 2', 'col 3']) df = df.to_json(orient='index') print("Json Encoded Array") print(df)

So What Do You Think?

I want to hear from you. What do you think of this article? Or maybe I missed one of the ways to serialize NumPy ndarray into JSON. Either way, let me know by leaving a comment below.

Also, try to solve the Python JSON Exercise to have a better understanding of Working with JSON Data in Python.

Did you find this page helpful? Let others know about it. Sharing helps me continue to create free Python resources.

About Vishal

I’m Vishal Hule, Founder of PYnative.com. I am a Python developer, and I love to write articles to help students, developers, and learners. Follow me on Twitter

Related Tutorial Topics:

Python Exercises and Quizzes

Free coding exercises and quizzes cover Python basics, data structure, data analytics, and more.

• 15+ Topic-specific Exercises and Quizzes
• Each Exercise contains 10 questions
• Each Quiz contains 12-15 MCQ

Источник

Convert NumPy array to JSON Python(5ways)

In this post, we are going to learn to Convert NumPy array to JSON Python with examples.We will use the JSON module that is imported using “import JSON“, First need to make sure the JSON module is installed or we can install it using “pip install jsonlib-python3” .NumPy land Pandas Library.

1. Convert NumPy array to JSON Python

In this Python exmple we are converting a Numpy array to JSON and storing it into JSON file.We have First converted NumPy array to list using tolist() method and iterate over the list using list comprehension along with enumerate() function.The data returned by list comprehension is used to write into JSON file using json.dump(data, Fileobj) method.

import numpy as np import json myarr = np.array([['Jack', 7, 44], ['rack', 13, 35], ['Sham', 13, 35]]) lists = myarr.tolist() data = [ for i, item in enumerate(lists)] print(data) with open('myJsonfile', 'w') as FileObj: json.dump(data, Fileobj)

2. Convert NumPy array to JSON Python

In this Python Program,We have define a NumpyEncoder(), The first parameter to json.dump() is Numpy array(myarr) and Second parameter is and we assign NumpyEncoder function to default=NumpyEncoder..Let us understand with Example

[["Jack", "7", "44"], ["rack", "13", "35"], ["Sham", "13", "35"]] JSON to NUmpy [['Jack', '7', '44'], ['rack', '13', '35'], ['Sham', '13', '35']]

3. Pandas to Convert NumPy array to JSON Python

We have Used Pandas libaray to_Json() method to convert NumPy array to JSON.First we have created a dataframe from NumPy array with Index and columns and use dataframe object to call the to_json() method with parameter orient=’index’

import pandas as pd import numpy myarr = np.array([['Jack', 7, 44],['rack', 13, 35],['Sham', 13, 35]]) dfobj = pd.DataFrame(myarr, index=['row 1', 'row 2','row 3'], columns=['Name', 'Age', 'Data']) dfobj = dfobj.to_json(orient='index') print("NumPy array to Json:") print(dfobj)

NumPy array to Json: ,"row 2":,"row 3":>

4. Pandas Convert NumPy array to JSON orient by Values

We have Used Pandas libaray to_Json() method to convert NumPy array to JSON.First we have created a dataframe from NumPy array with Index and columns and use dataframe object to call the to_json() method with parameter orient=’Values’ instead of index

import pandas as pd import numpy myarr = np.array([['Jack', 7, 44],['rack', 13, 35],['Sham', 13, 35]]) dfobj = pd.DataFrame(myarr, index=['row 1', 'row 2','row 3'], columns=['Name', 'Age', 'Data']) dfobj = dfobj.to_json(orient='values') print("NumPy array to Json:") print(dfobj)

NumPy array to Json: [["Jack","7","44"],["rack","13","35"],["Sham","13","35"]]

5. Pandas Convert NumPy array to JSON orient by Values

In this Python program example, We have used Pandas Library, First, we have installed it on the local system and imported using “”.We have Used the Pandas library to_Json() method to convert the NumPy array to JSON. First, we have created a data frame from NumPy array with Index and columns and use data frame object to call the to_json() method with parameter orient=’split’ instead of index and values

import pandas as pd import numpy myarr = np.array([['Jack', 7, 44],['rack', 13, 35],['Sham', 13, 35]]) dfobj = pd.DataFrame(myarr, index=['row 1', 'row 2','row 3'], columns=['Name', 'Age', 'Data']) dfobj = dfobj.to_json(orient='split') print("NumPy array to Json:") print(dfobj)

Summary

In this post, we have learned how to Convert the NumPy array to JSON Python by using the JSON module, NumPy, and Pandas.

Источник