Python convert bytes to json

Python — Convert a bytes array into JSON format

Your bytes object is almost JSON, but it’s using single quotes instead of double quotes, and it needs to be a string. So one way to fix it is to decode the bytes to str and replace the quotes. Another option is to use ast.literal_eval ; see below for details. If you want to print the result or save it to a file as valid JSON you can load the JSON to a Python list and then dump it out. Eg,

import json my_bytes_value = b'[]' # Decode UTF-8 bytes to Unicode, and convert single quotes # to double quotes to make it valid JSON my_json = my_bytes_value.decode('utf8').replace("'", '"') print(my_json) print('- ' * 20) # Load the JSON to a Python list & dump it back out as formatted JSON data = json.loads(my_json) s = json.dumps(data, indent=4, sort_keys=True) print(s) 

As Antti Haapala mentions in the comments, we can use ast.literal_eval to convert my_bytes_value to a Python list, once we’ve decoded it to a string.

from ast import literal_eval import json my_bytes_value = b'[]' data = literal_eval(my_bytes_value.decode('utf8')) print(data) print('- ' * 20) s = json.dumps(data, indent=4, sort_keys=True) print(s) 

Generally, this problem arises because someone has saved data by printing its Python repr instead of using the json module to create proper JSON data. If it’s possible, it’s better to fix that problem so that proper JSON data is created in the first place.

Solution 2

import json json.loads(my_bytes_value) 

Solution 3

Python 3.5 + Use io module

import json import io my_bytes_value = b'[]' fix_bytes_value = my_bytes_value.replace(b"'", b'"') my_json = json.load(io.BytesIO(fix_bytes_value)) 

Solution 4

To convert this bytesarray directly to json, you could first convert the bytesarray to a string with decode(), utf-8 is standard. Change the quotation markers.. The last step is to remove the » from the dumped string, to change the json object from string to list.

Solution 5

import json byte_array_example = b'' res = json.loads(byte_array_example.decode('unicode_escape')) print(res) 

decode by utf-8 cannot decode unicode characters. The right solution is uicode_escape

Источник

[python] Python — Convert a bytes array into JSON format

I want to convert a bytes array to JSON format. This is the source I have:

And this is the desired outcome I want to have:

First, I converted the bytes to string:

my_new_string_value = my_bytes_value.decode("utf-8") 

but when I try to loads to JSON:

my_json = json.loads(my_new_string_value) 

json.decoder.JSONDecodeError: Expecting value: line 1 column 174 (char 173) 

This question is related to python json

The answer is

Your bytes object is almost JSON, but it’s using single quotes instead of double quotes, and it needs to be a string. So one way to fix it is to decode the bytes to str and replace the quotes. Another option is to use ast.literal_eval ; see below for details. If you want to print the result or save it to a file as valid JSON you can load the JSON to a Python list and then dump it out. Eg,

import json my_bytes_value = b'[]' # Decode UTF-8 bytes to Unicode, and convert single quotes # to double quotes to make it valid JSON my_json = my_bytes_value.decode('utf8').replace("'", '"') print(my_json) print('- ' * 20) # Load the JSON to a Python list & dump it back out as formatted JSON data = json.loads(my_json) s = json.dumps(data, indent=4, sort_keys=True) print(s) 

As Antti Haapala mentions in the comments, we can use ast.literal_eval to convert my_bytes_value to a Python list, once we’ve decoded it to a string.

from ast import literal_eval import json my_bytes_value = b'[]' data = literal_eval(my_bytes_value.decode('utf8')) print(data) print('- ' * 20) s = json.dumps(data, indent=4, sort_keys=True) print(s) 

Generally, this problem arises because someone has saved data by printing its Python repr instead of using the json module to create proper JSON data. If it’s possible, it’s better to fix that problem so that proper JSON data is created in the first place.

import json json.loads(my_bytes_value) 

Источник

Python convert bytes to json

Last updated: Feb 22, 2023
Reading time · 3 min

# Table of Contents

# Convert Bytes to Dictionary in Python

To convert a bytes object to a dictionary:

1. Use the bytes.decode() method to convert the bytes to a string.
2. Use the ast.literal_eval() method to convert the string to a dictionary.
3. The literal_eval() method safely evaluates a string that contains a Python literal.

Copied! 
from ast import literal_eval my_bytes = b"" my_dict = literal_eval(my_bytes.decode('utf-8')) print(my_dict) # 👉️ print(type(my_dict)) # 👉️ 

We used the bytes.decode() method to convert the bytes object to a string.

Copied! 
my_bytes = b"" my_str = my_bytes.decode('utf-8') print(my_str) # 👉️ "" print(type(my_str)) # 👉️ 

The bytes.decode method returns a string decoded from the given bytes. The default encoding is utf-8 .

The last step is to pass the string to the ast.literal_eval() method.

The ast.literal_eval method allows us to safely evaluate a string that contains a Python literal.

Copied! 
from ast import literal_eval my_bytes = b"" my_dict = literal_eval(my_bytes.decode('utf-8')) print(my_dict) # 👉️ print(type(my_dict)) # 👉️ 

The string may consist of strings, bytes, numbers, tuples, lists, dictionaries, sets, booleans and None.

If the properties in your bytes object are wrapped in double quotes, you can also use the json.loads() method.

# Convert Bytes to Dictionary using json.loads()

This is a two-step process:

1. Use the bytes.decode() method to convert the bytes to a string.
2. Use the json.loads() method to parse the string into a dictionary.

Copied! 
import json my_bytes = b'' my_dict = json.loads(my_bytes.decode('utf-8')) print(my_dict) # 👉️ print(type(my_dict)) # 👉️ 

The json.loads method parses a JSON string into a native Python object.

Copied! 
import json json_str = r'' my_dict = json.loads(json_str) print(type(my_dict)) # 👉️ 

However, the method only accepts valid JSON strings.

If the properties in your bytes object are not wrapped in double quotes, use the ast.literal_eval() method.

# Replacing the single quotes with double quotes before using json.loads()

An alternative approach would be to replace the single quotes in the string with double quotes.

Copied! 
import json my_bytes = b"" my_str = my_bytes.decode('utf-8').replace("'", '"') print(my_str) my_dict = json.loads(my_str) print(my_dict) # 👉️ print(type(my_dict)) # 👉️ 

We used the str.replace() method to replace all single quotes in the string with double quotes.

However, this could go wrong if some of the values also contain single quotes.

The ast.literal_eval() method safely evaluates the string, so it should be your preferred approach.

# Converting a Dictionary to Bytes

If you need to convert a dictionary to bytes:

1. Use the json.dumps() method to convert the dictionary to a JSON string.
2. Use the str.encode() method to convert the string to bytes.

Copied! 
import json my_dict =  'first': 'bobby', 'last': 'hadz', 'age': 30, > my_str = json.dumps(my_dict) print(my_str) # 👉️ '' my_bytes = my_str.encode('utf-8') print(my_bytes) # 👉️ b'' 

The json.dumps method converts a Python object to a JSON formatted string.

Once we have a string, we can use the str.encode() method to convert it to bytes.

The str.encode method returns an encoded version of the string as a bytes object. The default encoding is utf-8 .

# Additional Resources

You can learn more about the related topics by checking out the following tutorials:

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.

Источник

Python convert bytes to json

Last updated: Jan 30, 2023
Reading time · 3 min

# TypeError: Object of type bytes is not JSON serializable

The Python «TypeError: Object of type bytes is not JSON serializable» occurs when we try to convert a bytes object to a JSON string.

To solve the error, call the decode() method on the bytes object to decode the bytes to a string before serializing to JSON.

Here is an example of how the error occurs.

Copied! 
import json my_bytes = 'bobbyhadz.com'.encode('utf-8') # ⛔️ TypeError: Object of type bytes is not JSON serializable json_str = json.dumps('message': my_bytes>) 

We tried passing a bytes object to the json.dumps() method but the method doesn’t handle bytes objects by default.

# Remove the call to encode to solve the error

To solve the error, either remove the call to the encode() method or use the decode() method to decode the bytes object to a string.

Copied! 
import json my_bytes = 'bobbyhadz.com'.encode('utf-8') # ✅ decode bytes object json_str = json.dumps('message': my_bytes.decode('utf-8')>) print(json_str) # 👉️ '' print(type(json_str)) # 👉️ 

The default JSON encoder handles str values, so we can decode the bytes object to a string before serializing to JSON.

The json.dumps method converts a Python object to a JSON formatted string.

The str.encode method returns an encoded version of the string as a bytes object. The default encoding is utf-8 .

The bytes.decode method returns a string decoded from the given bytes. The default encoding is utf-8 .

# Create a custom BytesEncoder class to solve the error

Alternatively, you can extend from the JSONEncoder class and handle the conversions in a default method.

Copied! 
import json class BytesEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, bytes): return obj.decode('utf-8') return json.JSONEncoder.default(self, obj) my_bytes = 'hello world'.encode('utf-8') json_str = json.dumps('message': my_bytes>, cls=BytesEncoder) print(json_str) # 👉️ '' print(type(json_str)) # 👉️ 

We extended from the JSONEncoder class.

The JSONEncoder class supports the following objects and types by default.

Python	JSON
dict	object
list, tuple	array
str	string
int, float, int and float derived Enums	number
True	true
False	false
None	null

Notice that the JSONEncoder class doesn’t support bytes to JSON conversion by default.

We can handle this by extending from the class and implementing a default() method that returns a serializable object.

Copied! 
import json class BytesEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, bytes): return obj.decode('utf-8') return json.JSONEncoder.default(self, obj) 

If the passed-in value is a bytes object, we decode it to a str and return the result.

The isinstance function returns True if the passed-in object is an instance or a subclass of the passed in class.

Источник