Web APIs, Python Requests & Performing an HTTP Request in Python Tutorial
Learn about the basics of HTTP and also about the request library in Python to make different types of requests.
Check out DataCamp’s Intermediate Importing Data in Python course that covers making HTTP requests.
Application Programming Interfaces (APIs) are software mediators ; their job is to permit applications to communicate with each other. These subtle mediators appear in everyday life whether you know it or not. For example, if you’ve sent an instant message today, you’ve used an API.
More specifically, APIs allow people to send and retrieve data using code. However, It’s more common to use APIs to retrieve data; for instance, you can read this blog post because your web browser retrieved the data that makes up this page from the DataCamp server.
But web servers don’t randomly send data — that would be like going to a restaurant, and the waiter randomly brings you a meal. A request must be made to the server to retrieve data before it responds with data. This is true for the waiter in a restaurant, and if you’d like to retrieve some data from an API – you make an API request to a server, and it will respond with the appropriate data.
The de facto industry standard for sending HTTP requests in Python is the requests library. There is also Python’s built-in urllib , but Pythonistas tend to prefer the python requests API due to its readability and the fact it supports fully restful APIs – something we will touch on a little later.
The requests library isolates all of the challenges of making requests behind a straightforward API — this allows you to concentrate on communicating with services and consuming data in your application.
In this article, we will walk through some of the core components of the requests library and provide some code examples to help you get started.
Requests: HTTP for Humans™¶
Requests is an elegant and simple HTTP library for Python, built for human beings.
Behold, the power of Requests:
>>> r = requests.get(‘https://api.github.com/user’, auth=(‘user’, ‘pass’)) >>> r.status_code 200 >>> r.headers[‘content-type’] ‘application/json; charset=utf8’ >>> r.encoding ‘utf-8’ >>> r.text ‘ >>> r.json()
Requests allows you to send HTTP/1.1 requests extremely easily. There’s no need to manually add query strings to your URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, thanks to urllib3.
Beloved Features¶
Requests is ready for today’s web.
- Keep-Alive & Connection Pooling
- International Domains and URLs
- Sessions with Cookie Persistence
- Browser-style SSL Verification
- Automatic Content Decoding
- Basic/Digest Authentication
- Elegant Key/Value Cookies
- Automatic Decompression
- Unicode Response Bodies
- HTTP(S) Proxy Support
- Multipart File Uploads
- Streaming Downloads
- Connection Timeouts
- Chunked Requests
- .netrc Support
Requests officially supports Python 3.7+, and runs great on PyPy.
The User Guide¶
This part of the documentation, which is mostly prose, begins with some background information about Requests, then focuses on step-by-step instructions for getting the most out of Requests.
- Installation of Requests
- $ python -m pip install requests
- Get the Source Code
- Make a Request
- Passing Parameters In URLs
- Response Content
- Binary Response Content
- JSON Response Content
- Raw Response Content
- Custom Headers
- More complicated POST requests
- POST a Multipart-Encoded File
- Response Status Codes
- Response Headers
- Cookies
- Redirection and History
- Timeouts
- Errors and Exceptions
- Session Objects
- Request and Response Objects
- Prepared Requests
- SSL Cert Verification
- Client Side Certificates
- CA Certificates
- Body Content Workflow
- Keep-Alive
- Streaming Uploads
- Chunk-Encoded Requests
- POST Multiple Multipart-Encoded Files
- Event Hooks
- Custom Authentication
- Streaming Requests
- Proxies
- Compliance
- HTTP Verbs
- Custom Verbs
- Link Headers
- Transport Adapters
- Blocking Or Non-Blocking?
- Header Ordering
- Timeouts
- Basic Authentication
- Digest Authentication
- OAuth 1 Authentication
- OAuth 2 and OpenID Connect Authentication
- Other Authentication
- New Forms of Authentication
The Community Guide¶
This part of the documentation, which is mostly prose, details the Requests ecosystem and community.
- Recommended Packages and Extensions
- Certifi CA Bundle
- CacheControl
- Requests-Toolbelt
- Requests-Threads
- Requests-OAuthlib
- Betamax
- Encoded Data?
- Custom User-Agents?
- Why not Httplib2?
- Python 3 Support?
- Python 2 Support?
- What are “hostname doesn’t match” errors?
- Python for iOS
- Stack Overflow
- File an Issue
- Send a Tweet
- Process
- Previous CVEs
- Major Releases
- Minor Releases
- Hotfix Releases
- Reasoning
The API Documentation / Guide¶
If you are looking for information on a specific function, class, or method, this part of the documentation is for you.
- Developer Interface
- Main Interface
- Exceptions
- Request Sessions
- Lower-Level Classes
- Lower-Lower-Level Classes
- Authentication
- Encodings
- Cookies
- Status Code Lookup
- Migrating to 1.x
- Migrating to 2.x
The Contributor Guide¶
If you want to contribute to the project, this part of the documentation is for you.
- Contributor’s Guide
- Be Cordial
- Get Early Feedback
- Contribution Suitability
- Code Contributions
- Steps for Submitting Code
- Code Review
- Code Style
- New Contributors
- Keepers of the Crystals
- Previous Keepers of Crystals
- Patches and Suggestions
There are no more guides. You are now guideless. Good luck.
Requests is an elegant and simple HTTP library for Python, built for human beings.
Useful Links
