Python requests content type json

How to send a POST with Python Requests?

A POST request is a particular type of HTTP method used when we send data to services on the web. We use them on web sites that use forms — when we login, when we send messages or post an image. Applications also use POST requests to interact with other services to send data with JSON being a common data format for exchange.

The Requests library is one of the most popular HTTP client libraries for Python. It currently has over 45k stars on Github, with downloads on PyPI of 115M a month! It makes sending POST requests much simpler programmatically than having to send data via a headless browser. With a headless browser we’d need to write scripts to navigate to a site, move between fields, populate each of them with the desired data before finally submitting the data. By sending a POST request, we skip straight to the final submission step. Requests also is a much, much smaller library than a browser resulting in better performance and memory usage.

In this article we’ll cover how to construct a POST request using Requests and how it can make the process much simpler for us.

Building a JSON POST Request with Requests

As an example, lets start by building a JSON POST request the hard way. Don’t worry Requests will simplify this for us later! We’re using the httpbin.org service, which returns a JSON response detailing the content that was sent.

1. Set the Request Method to POST

Requests has a really simple HTTP verb based design, meaning that to get a response we call it’s .post() method supplying our URI as an argument. It also provides similar methods for GET, PUT and PATCH requests.

import requests r = requests.post("https://httpbin.org/post") 

2. Set the POST data

To actually send some data, we supply a data argument. By default data is sent using as a HTML form as we would get by submitting any form on the web. Requests sets the content type to ‘application/x-www-form-urlencoded’, so it isn’t necessary to set any headers.

import requests import json r = requests.post("https://httpbin.org/post", data="key": "value">, ) 

Inspecting the response from the httpbin.org service, we can see what was sent as form data under the «form» key.

>>> r.text '\n "args": <>, \n "data": "", \n "files": <>, \n "form": \n "key": "value"\n >, \n "headers": \n "Accept": "*/*", \n "Accept-Encoding": "gzip, deflate", \n "Content-Length": "9", \n "Content-Type": "application/x-www-form-urlencoded", \n "Host": "httpbin.org", \n "User-Agent": "python-requests/2.25.1", \n "X-Amzn-Trace-Id": "Root=1-60df1a04-0384d3ce7d9ac00b5855064b"\n >, \n "json": null, \n "origin": "**.***.**.***", \n "url": "https://httpbin.org/post"\n>\n' 

If we want to send JSON, we can supply a JSON formatted string. In our example, we’re using the json module to convert a dictionary to JSON formatted string ready to send.

import requests import json r = requests.post("https://httpbin.org/post", data=json.dumps("key": "value">), ) 

3. Set the POST Headers

Our JSON example isn’t going to work as it is now. With many services, we’ll likely get a 400 (Bad Request) HTTP status code. To prevent this we also need to inform the service we’re calling that our data is JSON so it can be handled correctly. To do so, we set the ‘Content-Type’ to ‘application/json’ in the request headers:

import requests import json r = requests.post( "https://httpbin.org/post", data=json.dumps("key": "value">), headers="Content-Type": "application/json">, ) 

4. POST JSON Data

If you think our JSON examples so far look a bit complex — you’re totally right. Requests makes it very easy to reduce all this down to a much simpler call. We can just supply a ‘json’ argument with our data. Requests will correctly set our headers and encode the JSON formatted string for us automatically.

import requests r = requests.post('https://httpbin.org/post', json='key':'value'>) 

Reading JSON Responses

By inspecting the response from our service, we can see the data returned is a JSON formatted string too. This is only text, so we’d need to parse it ourselves to use it within our Python script.

>>> r.text '\n "args": <>, \n "data": "\\"key\\": \\"value\\">", \n "files": <>, \n "form": <>, \n "headers": \n "Accept": "*/*", \n "Accept-Encoding": "gzip, deflate", \n "Content-Length": "16", \n "Content-Type": "application/json", \n "Host": "httpbin.org", \n "User-Agent": "python-requests/2.25.1", \n "X-Amzn-Trace-Id": "Root=1-60df0aaa-3105fc35436042571335fa22"\n >, \n "json": \n "key": "value"\n >, \n "origin": "**.***.**.***", \n "url": "https://httpbin.org/post"\n>\n' 

Alternatively, if we want to access the response as a JSON object we can also use Requests built in JSON decoder by calling .json() on the response object.

>>> r.json() 'args': <>, 'data': '', 'files': <>, 'form': <>, 'headers': 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '16', 'Content-Type': 'application/json', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.25.1', 'X-Amzn-Trace-Id': 'Root=1-60deee08-4e9c76b6457826d5001b76fa'>, 'json': 'key': 'value'>, 'origin': '**.***.**.***', 'url': 'http://httpbin.org/post'> 

Making POST requests within a Session

As a final example, lets login to Hacker News using Requests and submit some form data to login in combination with a requests.Session() object. Any cookies we receive will be stored in the object, and will be reused in later .get() calls to the session.

For our example, we’ll search the response text of the news page to confirm if we’re shown the logout link indicating we’re being served a logged in page.

import requests session = requests.Session() r = session.post("https://news.ycombinator.com/login", data=< "acct": "username", "pw": "***not-telling***" >) r = session.get("https://news.ycombinator.com/news") logout_pos = r.text.find("logout") print(r.text[logout_pos-20:logout_pos+20]) # 

Great! Our link is in the response indicating we’re logged in. We can now continue to interact with the site using the session object via Requests.

Conclusion

You’ve seen just how simple Requests makes it to post data to sites or services and how we can reduce much of the common code in our applications by using it. This simple design has certainly contributed to it’s success in the Python community, making it a firm favorite with developers. Before you reach for a full blown headless browser to send your data, you should definitely consider using Requests.

You will often need proxies for your web scraping projects, don’t hesitate to checkout this article on how to use proxies with Python Requests.

Ian is a freelance developer with a passion for simple solutions. He has written code to power surveys, studio pipelines and holds a PhD in distributed computing.

Источник

Python Post JSON using requests library

In this article, I will let you know how to post a JSON from a client to a server using a requests library. Also, if you are facing a “400 bad request error” while posting JSON to the server, this article will try to solve that.

Further Reading:

Steps to Build a JSON POST request

Create a URL object: Let’s create a URL object. We need a target URI string that accepts the JSON data via HTTP POST method. In this example, I am using httpbin.org service to Post JSON data. httpbin.org is a web service that allows us to test the HTTP request. You can use it to test and inspect your POST request. httpbin.org responds with data about your request.

So my URL is: “https://httpbin.org/post“

Set the Request Method: As the name suggests, we need to use a post method of a request module.

requests.post('https://httpbin.org/post')

Specify the POST data: As per the HTTP specification for a POST request, we pass data through the message body. Using requests, you’ll pass the payload to the corresponding function’s data parameter. Data can be anything including JSON, dictionary, a list of tuples, bytes, or a file-like object. In this example, I am sending the following JSON data.

If you have data in the form of a dictionary or any Python object, you can convert it into JSON like this.

import json sampleDict = < "id": 1, "name":"Jessa" >jsonData = json.dumps(sampleDict)

Use The json parameter: The requests module provides a json parameter that we can use to specify JSON data in the POST method. i.e., To send JSON data, we can also use the json parameter of the requests.post() method.

requests.post('https://httpbin.org/post', json=)

Why set it to json? Because it will help the request module to serialize your data into the JSON format. Now, Let’s see the example.

Approach 1: Using json parameter

import requests response = requests.post('https://httpbin.org/post', json=) print("Status code: ", response.status_code) print("Printing Entire Post Request") print(response.json())

Status code: 200 Printing Entire Post Request , 'data': '', 'files': <>, 'form': <>, 'headers': , ' json': , 'origin': 'xxx.xx.xx.xx, xxx.xx.xx.xx', 'url': 'https://httpbin.org/post'>

Note: This service returns your entire request as a response so it will help you to know details about your request.

Approach 2: By setting header information

Alternatively, we can set the request’s content-type. In this example, we are passing JSON, so the request’s content type is application/json .

By specifying correct request headers so that the requests module can serialize your data into the correct Content-Type header format. In this can we don’t need to use the json parameter. This is useful for an older version. Let’s see the example now.

import requests newHeaders = response = requests.post('https://httpbin.org/post', data=, headers=newHeaders) print("Status code: ", response.status_code) response_Json = response.json() print("Printing Post JSON data") print(response_Json['data']) print("Content-Type is ", response_Json['headers']['Content-Type'])

Status code: 200 Printing Post JSON data id=1&name=Jessa application/json

Test Your JSON POST request using postman before executing

It is always a best practice to test your request along with its message body using postman to verify JSON data, and a request is in the required format. Let’s see how to test POST request using postman.

Add Postman extension or install a native postman app. Let’s see the steps now.

• Select POST request and enter your service POST operation URL.
• Click on Headers. In the key column enter Content-Type and in the Value column enter application/json .
• Click on the body section and click the raw radio button. enter your JSON data. Click the Send button.

Did you find this page helpful? Let others know about it. Sharing helps me continue to create free Python resources.

About Vishal

I’m Vishal Hule, Founder of PYnative.com. I am a Python developer, and I love to write articles to help students, developers, and learners. Follow me on Twitter

Related Tutorial Topics:

Python Exercises and Quizzes

Free coding exercises and quizzes cover Python basics, data structure, data analytics, and more.

• 15+ Topic-specific Exercises and Quizzes
• Each Exercise contains 10 questions
• Each Quiz contains 12-15 MCQ

Источник