Python read JSON file and modify
Hi I am trying to take the data from a json file and insert and id then perform POST REST. my file data.json has:
import json f = open("data.json","r") data = f.read() jsonObj = json.loads(data) I can’t get to load the json format file. What should I do so that I can convert the json file into json object and add another id value.
7 Answers 7
import json with open('data.json', 'r+') as f: data = json.load(f) data['id'] = 134 # unrelated: json format is defined for Unicode text. You could use with codecs.open('data.json', 'r+', encoding='utf-8') as f
@VladimBelov answer is better in my opinion. Though It doesn't need the os.remove(filename) because opening the file in 'w' (write) mode will truncate as necessary.
@alper, You need only single f.seek(0) call after the last d['. '] = . . f.seek(0); json.dump(. ); f.truncate() write complete file all over again.
@alper, (1) Inserting something in file requires rewrite after the position. (2) It's hard to know where the insertion position will be (depending on json serialization implementation.)
@Kalcifer, It is required. especially when new id's representation is shorter than old one. And, json.dump() does not call truncate according to code
falsetru's solution is nice, but has a little bug:
Suppose original 'id' length was larger than 5 characters. When we then dump with the new 'id' (134 with only 3 characters) the length of the string being written from position 0 in file is shorter than the original length. Extra chars (such as '>') left in file from the original content.
I solved that by replacing the original file.
import json import os filename = 'data.json' with open(filename, 'r') as f: data = json.load(f) data['id'] = 134 # I would like to present a modified version of Vadim's solution. It helps to deal with asynchronous requests to write/modify json file. I know it wasn't a part of the original question but might be helpful for others.
In case of asynchronous file modification os.remove(filename) will raise FileNotFoundError if requests emerge frequently. To overcome this problem you can create temporary file with modified content and then rename it simultaneously replacing old version. This solution works fine both for synchronous and asynchronous cases.
import os, json, uuid filename = 'data.json' with open(filename, 'r') as f: data = json.load(f) data['id'] = 134 # due to this issue it would be better to use os.replace(tempfile, filename) for a cross-platform solution
There is really quite a number of ways to do this and all of the above are in one way or another valid approaches. Let me add a straightforward proposition. So assuming your current existing json file looks is this.
And you want to bring in this new json content (adding key "id")
My approach has always been to keep the code extremely readable with easily traceable logic. So first, we read the entire existing json file into memory, assuming you are very well aware of your json's existing key(s).
import json # first, get the absolute path to json file PATH_TO_JSON = 'data.json' # assuming same directory (but you can work your magic here with os.) # read existing json to memory. you do this to preserve whatever existing data. with open(PATH_TO_JSON,'r') as jsonfile: json_content = json.load(jsonfile) # this is now in memory! you can use it outside 'open' Next, we use the 'with open()' syntax again, with the 'w' option. 'w' is a write mode which lets us edit and write new information to the file. Here s the catch that works for us . any existing json with the same target write name will be erased automatically.
So what we can do now, is simply write to the same filename with the new data
# add the id key-value pair (rmbr that it already has the "name" key value) json_content["id"] = "134" with open(PATH_TO_JSON,'w') as jsonfile: json.dump(json_content, jsonfile, indent=4) # you decide the indentation level And there you go! data.json should be good to go for an good old POST request
Loading a JSON File in Python – How to Read and Parse JSON
Dillion Megida
In this article, you'll learn how to read and parse JSON in Python.
What is JSON?
JSON is short for JavaScript Object Notation. It's a simple syntax for storing data in name-value pairs. Values can be different data types as long as they are valid. Non-acceptable types for JSON include functions, dates, and undefined .
JSON files are stored with the .json extension with a valid JSON structure.
Here's what the structure of a JSON file looks like:
You'll often use JSON to send and receive data from a server in web applications.
When the data is received, the program reads and parses the JSON to extract specific data. Different languages have their own methods for doing this. We'll look at how to do these in Python here.
How to Read JSON Files
Let's say the JSON in the code block above is stored in a user.json file. Using the open() inbuilt function in Python, we can read that file and assign the content to a variable. Here's how:
with open('user.json') as user_file: file_contents = user_file.read() print(file_contents) # You pass the file path to the open method which opens the file and assigns the stream data from the file to the user_file variable. Using the read method, you can pass the text contents of the file to the file_contents variable.
I used with at the beginning of the expression so that after reading the contents of the file, Python can close the file.
file_contents now contains a stringified version of the JSON. As a next step, you can now parse the JSON.
How to Parse JSON
Python has in-built modules for various operations. For managing JSON files, Python has the json module.
This module comes with many methods. One of which is the loads() method for parsing JSON strings. Then, you can assign the parsed data to a variable like this:
import json with open('user.json') as user_file: file_contents = user_file.read() print(file_contents) parsed_json = json.loads(file_contents) # Using the loads() method, you can see that the parsed_json variable now has a valid dictionary. From this dictionary, you can access the keys and values in it.
Also notice how null from the JSON is converted to None in python. This is because null is not valid in Python .
How to Use json.load() to Read and Parse JSON Files
The json module also has the load method which you can use to read a file object and parse it at the same time. Using this method, you can update the previous code to this:
import json with open('user.json') as user_file: parsed_json = json.load(user_file) print(parsed_json) # Instead of using the read method of the file object and using the loads method of the json module, you can directly use the load method which reads and parses the file object.
Wrapping Up
JSON data is commonly known for its simple structure and is popular (a standard in most cases) for information exchange between servers and clients.
Different languages and technologies can read and parse JSON files in different ways. In this article, we've learned how to read JSON files and parse such files using the read method of file objects, and the loads and load methods of the json module.
Dillion Megida
Developer Advocate and Content Creator passionate about sharing my knowledge on Tech. I simplify JavaScript / ReactJS / NodeJS / Frameworks / TypeScript / et al My YT channel: youtube.com/c/deeecode
If you read this far, tweet to the author to show them you care. Tweet a thanks
Learn to code for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started
freeCodeCamp is a donor-supported tax-exempt 501(c)(3) charity organization (United States Federal Tax Identification Number: 82-0779546)
Our mission: to help people learn to code for free. We accomplish this by creating thousands of videos, articles, and interactive coding lessons - all freely available to the public. We also have thousands of freeCodeCamp study groups around the world.
Donations to freeCodeCamp go toward our education initiatives, and help pay for servers, services, and staff.
How can I parse (read) and use JSON in Python?
My Python program receives JSON data, and I need to get bits of information out of it. How can I parse the data and use the result? I think I need to use json.loads for this task, but I can't understand how to do it. For example, suppose that I have jsonStr = '<"one" : "1", "two" : "2", "three" : "3">' . Given this JSON, and an input of "two" , how can I get the corresponding data, "2" ? Beware that .load is for files; .loads is for strings. See also: Reading JSON from a file. Occasionally, a JSON document is intended to represent tabular data. If you have something like this and are trying to use it with Pandas, see Python - How to convert JSON File to Dataframe. Some data superficially looks like JSON, but is not JSON. For example, sometimes the data comes from applying repr to native Python data structures. The result may use quotes differently, use title-cased True and False rather than JSON-mandated true and false , etc. For such data, see Convert a String representation of a Dictionary to a dictionary or How to convert string representation of list to a list. Another common variant format puts separate valid JSON-formatted data on each line of the input. (Proper JSON cannot be parsed line by line, because it uses balanced brackets that can be many lines apart.) This format is called JSONL. See Loading JSONL file as JSON objects. Sometimes JSON data from a web source is padded with some extra text. In some contexts, this works around security restrictions in browsers. This is called JSONP and is described at What is JSONP, and why was it created?. In other contexts, the extra text implements a security measure, as described at Why does Google prepend while(1); to their JSON responses?. Either way, handling this in Python is straightforward: simply identify and remove the extra text, and proceed as before.