Python post files and json

Python Post JSON using requests library

In this article, I will let you know how to post a JSON from a client to a server using a requests library. Also, if you are facing a “400 bad request error” while posting JSON to the server, this article will try to solve that.

Further Reading:

Steps to Build a JSON POST request

Create a URL object: Let’s create a URL object. We need a target URI string that accepts the JSON data via HTTP POST method. In this example, I am using httpbin.org service to Post JSON data. httpbin.org is a web service that allows us to test the HTTP request. You can use it to test and inspect your POST request. httpbin.org responds with data about your request.

So my URL is: “https://httpbin.org/post“

Set the Request Method: As the name suggests, we need to use a post method of a request module.

requests.post('https://httpbin.org/post')

Specify the POST data: As per the HTTP specification for a POST request, we pass data through the message body. Using requests, you’ll pass the payload to the corresponding function’s data parameter. Data can be anything including JSON, dictionary, a list of tuples, bytes, or a file-like object. In this example, I am sending the following JSON data.

If you have data in the form of a dictionary or any Python object, you can convert it into JSON like this.

import json sampleDict = < "id": 1, "name":"Jessa" >jsonData = json.dumps(sampleDict)

Use The json parameter: The requests module provides a json parameter that we can use to specify JSON data in the POST method. i.e., To send JSON data, we can also use the json parameter of the requests.post() method.

requests.post('https://httpbin.org/post', json=)

Why set it to json? Because it will help the request module to serialize your data into the JSON format. Now, Let’s see the example.

Approach 1: Using json parameter

import requests response = requests.post('https://httpbin.org/post', json=) print("Status code: ", response.status_code) print("Printing Entire Post Request") print(response.json())

Status code: 200 Printing Entire Post Request , 'data': '', 'files': <>, 'form': <>, 'headers': , ' json': , 'origin': 'xxx.xx.xx.xx, xxx.xx.xx.xx', 'url': 'https://httpbin.org/post'>

Note: This service returns your entire request as a response so it will help you to know details about your request.

Approach 2: By setting header information

Alternatively, we can set the request’s content-type. In this example, we are passing JSON, so the request’s content type is application/json .

By specifying correct request headers so that the requests module can serialize your data into the correct Content-Type header format. In this can we don’t need to use the json parameter. This is useful for an older version. Let’s see the example now.

import requests newHeaders = response = requests.post('https://httpbin.org/post', data=, headers=newHeaders) print("Status code: ", response.status_code) response_Json = response.json() print("Printing Post JSON data") print(response_Json['data']) print("Content-Type is ", response_Json['headers']['Content-Type'])

Status code: 200 Printing Post JSON data id=1&name=Jessa application/json

Test Your JSON POST request using postman before executing

It is always a best practice to test your request along with its message body using postman to verify JSON data, and a request is in the required format. Let’s see how to test POST request using postman.

Add Postman extension or install a native postman app. Let’s see the steps now.

• Select POST request and enter your service POST operation URL.
• Click on Headers. In the key column enter Content-Type and in the Value column enter application/json .
• Click on the body section and click the raw radio button. enter your JSON data. Click the Send button.

Did you find this page helpful? Let others know about it. Sharing helps me continue to create free Python resources.

About Vishal

I’m Vishal Hule, Founder of PYnative.com. I am a Python developer, and I love to write articles to help students, developers, and learners. Follow me on Twitter

Related Tutorial Topics:

Python Exercises and Quizzes

Free coding exercises and quizzes cover Python basics, data structure, data analytics, and more.

• 15+ Topic-specific Exercises and Quizzes
• Each Exercise contains 10 questions
• Each Quiz contains 12-15 MCQ

Источник

Python Requests: Post JSON and file in single request

I need to do a API call to upload a file along with a JSON string with details about the file. I am trying to use the python requests lib to do this:

import requests info = < 'var1' : 'this', 'var2' : 'that', >data = json.dumps(< 'token' : auth_token, 'info' : info, >) headers = files = r = requests.post(url, files=files, data=data, headers=headers) 

 raise ValueError("Data must not be a string.") ValueError: Data must not be a string 

If I remove the ‘files’ from the request, it works.
If I remove the ‘data’ from the request, it works.
If I do not encode data as JSON it works. For this reason I think the error is to do with sending JSON data and files in the same request. Any ideas on how to get this working?

7 Answers 7

Do not set the Content-type header yourself, leave that to pyrequests to generate

def send_request(): payload = files = < 'json': (None, json.dumps(payload), 'application/json'), 'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream') >r = requests.post(url, files=files) print(r.content) 

@DanSalo thanks, i forgot to answer, I did find it a will. thanks for your confirmation ( upvoted your comments )

import requests info = < 'var1' : 'this', 'var2' : 'that', >data = < 'token' : auth_token, 'info' : info, >headers = files = r = requests.post(url, files=files, data=data, headers=headers) 

Note that this may not necessarily be what you want, as it will become another form-data section.

If I do as you suggest, I get another exception: «need more than 1 value to unpack» and wonder what to do with it 🙁

This will only work if data are simple key-value pairs (form parameters-like), but all nested stuff will be truncated as HTTP form-encoding is not capable of representing nested data structures.

Thanks, @hoefling. You’ve saved my life. I was trying to understand for 1 hour why the hell that library truncates it (or what was happening).

@proteneer This does not seems to be working for me, Doing a call like . d = requests.post(‘http://localhost:18090/upload’,files=< 'face_image': ('mama_justkilled_aman.jpg', open('mama_justkilled_aman.jpg', 'rb'), 'image/jpeg' ) >, data=< 'gffp': 42 >) but data is not being received by the server

I’m don’t think you can send both data and files in a multipart encoded file, so you need to make your data a «file» too:

files = < 'data' : data, 'document': open('file_name.pdf', 'rb') >r = requests.post(url, files=files, headers=headers) 

To note: on the receiving end: the request.files[‘data’] is a fileStorage tuple. What needs to be done is do a request.files[‘data’].read() to get the actual data (which is a json-encoded string) so you’ll need to do something like json.loads(request.files[‘data’].read())

I have been using requests==2.22.0

For me , the below code worked.

import requests data = < 'var1': 'this', 'var2': 'that' >r = requests.post("http://api.example.com/v1/api/some/", files=, data=data, headers=. #since I had to authenticate for the same ) print (r.json()) 

For sending Facebook Messenger API, I changed all the payload dictionary values to be strings. Then, I can pass the payload as data parameter.

import requests ACCESS_TOKEN = '' url = 'https://graph.facebook.com/v2.6/me/messages' payload = < 'access_token' : ACCESS_TOKEN, 'messaging_type' : "UPDATE", 'recipient' : '', 'message' : '>>', > files = r = requests.post(url, files=files, data=payload) 

1. Sending request

import json import requests cover = 'superneat.jpg' payload = files = [ ('json', ('payload.json', json.dumps(payload), 'application/json')), ('cover', (cover, open(cover, 'rb'))) ] r = requests.post("https://superneatech.com/store/series", files=files) print(r.text) 

2. Receiving request

You will receive the JSON data as a file, get the content and continue.

That will only work if your file is at the same directory where your script is.

If you want to append file from different directory you should do:

Where dir_path is a directory with your ‘file_name.pdf’ file.

But what if you’d like to send multiple PDFs ?

You can simply make a custom function to return a list of files you need (in your case that can be only those with .pdf extension). That also includes files in subdirectories (search for files recursively):

def prepare_pdfs(): return sorted([os.path.join(root, filename) for root, dirnames, filenames in os.walk(dir_path) for filename in filenames if filename.endswith('.pdf')]) 

for file in my_data: pdf = open(file, 'rb') files = < 'document': pdf >r = requests.post(url, files=files, . ) 

Источник

how to POST contents of JSON file to RESTFUL API with Python using requests module

Okay, I give up. I am trying to post the contents of a file that contains JSON. The contents of the file look like this:

#!/usr/bin/env python3 import requests import json files = headers = r = requests.post('https://api.example.com/api/dir/v1/accounts/9999999/orders', files=files, headers=headers) 

4 Answers 4

This should work, but it’s meant for very large files.

import requests url = 'https://api.example.com/api/dir/v1/accounts/9999999/orders' headers = r = requests.post(url, data=open('example.json', 'rb'), headers=headers) 

If you want to send a smaller file, send it as a string.

contents = open('example.json', 'rb').read() r = requests.post(url, data=contents, headers=headers) 

Here, headers = <'Authorization' : ‘(some auth code)’ . >‘(some auth code)’ should be replaced by ‘auth = (‘username’,’password’)’ is that correct ?

More likely ‘(some auth code)’ = ‘(username, password)’. But I’m not actually sure how he’s authenticating with his server. It’s probably easier to pass that stuff as a keyword argument in the requests.post call, as in the docs: docs.python-requests.org/en/master/user/authentication

First of all your json file does not contain valid json. as in, «id” -here the closing quotation mark is different than the opening quotation mark. And other ID fields have the same error. Make it like this «id» .

now you can do it like this,

import requests import json with open('example.json') as json_file: json_data = json.load(json_file) headers = r = requests.post('https://api.example.com/api/dir/v1/accounts/9999999/orders', data=json.dumps(json_data), headers=headers) 

Источник

post-ing a file using python requests and json

I’ve been given the following curl command as part of API documentation, and I’m trying to implement it using the requests library.

curl -v --cookie cookie.txt -X POST -H 'Accept: application/json' -F 'spot[photo]'=@rails.png -F 'spot[description]'=spot_description -F 'spot[location_id]'=9 -F 'spot[categories][]'='See the Sights' -F 'spot[categories][]'='Learn Something' http://some.server.com/api/v1/spots 

import requests import json _user = 'redacted' _password = 'redacted' _session = requests.session() _server = 'http://some.server.com' _hdr = _login_payload = < 'user': < 'email': _user, 'password': _password >> r = _session.post(_server + "/users/sign_in", data=json.dumps(_login_payload), headers=_hdr) print json.loads(r.content) _spot_payload = < 'spot': < 'photo': '@rails.gif', 'description': 'asdfghjkl', 'location_id': 9, 'categories': ['See the Sights',] >> r = _session.post(_server + '/api/v1/spots', data=json.dumps(_spot_payload), headers=_hdr) print json.loads(r.content) 

I’ve heard tell that you can use open(‘file’).read() to post files, but the json encoder doesn’t much like this, and I’m not sure about a way around it.

1 Answer 1

When you issue this command:

C:\>curl -X POST -H "Accept:application/json" -F "spot[photo]=@file.txt" -F "spot[description]=spot_description" http://localhost:8888 

what’s being sent looks like this:

POST / HTTP/1.1 User-Agent: curl/7.25.0 (i386-pc-win32) libcurl/7.25.0 OpenSSL/0.9.8u zlib/1.2.6 libssh2/1.4.0 Host: localhost:8888 Accept: application/json Content-Length: 325 Expect: 100-continue Content-Type: multipart/form-data; boundary=—————————-e71aebf115cd

——————————e71aebf115cd Content-Disposition: form-data; name=»spot[photo]»; filename=»file.txt» Content-Type: text/plain

Some text. ——————————e71aebf115cd Content-Disposition: form-data; name=»spot[description]»

spot_description ——————————e71aebf115cd—

As you can see curl sends request with Content-Type set to multipart/form-data; Requests support sending files using the same Content-Type . You should use files argument for this.

(2.7) C:\>python Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> requests.__version__ '0.11.1' >>> requests.post('http://localhost:8888', files=, data=)

And what’s being sent looks like this:

POST http://localhost:8888/ HTTP/1.1 Host: localhost:8888 Content-Length: 342 Content-Type: multipart/form-data; boundary=192.168.1.101.1.8000.1334865122.004.1 Accept-Encoding: identity, deflate, compress, gzip Accept: */* User-Agent: python-requests/0.11.1 --192.168.1.101.1.8000.1334865122.004.1 Content-Disposition: form-data; name="spot[description]" Content-Type: text/plain spot_description --192.168.1.101.1.8000.1334865122.004.1 Content-Disposition: form-data; name="spot[photo]"; filename="file.txt" Content-Type: text/plain Some text. --192.168.1.101.1.8000.1334865122.004.1-- 

Источник