Python http requests cookies

Python http requests cookies

Last updated: Jul 2, 2023
Reading time · 4 min

# Table of Contents

# Python: How to get and set Cookies when using Requests

Use the Session class to set and get cookies when using the requests module in Python.

The class creates a Session object that stores the cookies and all requests that are made handle cookies automatically.

Copied! 
import requests session = requests.Session() print(session.cookies.get_dict()) # <> response = session.get('http://google.com') # print(session.cookies.get_dict()) 

Make sure you have the requests module installed to be able to run the code sample.

Copied! 
pip install requests # or with pip3 pip3 install requests 

The Session object enables you to persist cookies across requests.

The object persists cookies across all requests that were made using the Session instance.

The Session object has all of the methods of the main requests API.

The get_dict() method returns a Python dictionary of the name-value pairs of cookies.

You can also use Sessions as context managers.

Copied! 
import requests with requests.Session() as session: print(session.cookies.get_dict()) # <> response = session.get('http://google.com') # print(session.cookies.get_dict()) 

Make sure the code that accesses the Session object is inside the indented block.

# Accessing the path and the domain when using Cookies with requests

It is very likely, that you will also have to access the domain and the path to which the request was made when accessing the cookies.

You can use a list comprehension to construct a list of dictionaries that contain the path and domain.

Copied! 
import requests session = requests.Session() print(session.cookies.get_dict()) # <> print('-' * 50) response = session.get('http://google.com') # print(session.cookies.get_dict()) print('-' * 50) result = [ 'name': c.name, 'value': c.value, 'domain': c.domain, 'path': c.path> for c in session.cookies ] # [] print(result) 

We used a list comprehension to iterate over the RequestCookieJar object ( session.cookies ) and returned a dictionary on each iteration.

The dictionary contains the name and value of the cookie, the path and the domain.

# Sending cookies with a request

If you want to send cookies with a request, set the cookies keyword argument.

Copied! 
import requests session = requests.Session() response = session.get( 'https://httpbin.org/cookies', cookies='my-cookie': 'my-value'> ) # # "cookies": # "my-cookie": "my-value" # > # > print(response.text) 

We set the cookies keyword argument to a dictionary of key-value pairs.

# Accessing the cookies attribute directly on the Response object

In more recent versions of the requests module, you can also access the cookies attribute directly on the Response object.

Copied! 
import requests response = requests.get('http://google.com', timeout=30) # print(response.cookies.get_dict()) result = [ 'name': c.name, 'value': c.value, 'domain': c.domain, 'path': c.path> for c in response.cookies ] # [] print(result) 

Notice that we didn’t instantiate the Session class.

We simply accessed the cookies attribute on the Response object and called the get_dict() method on the RequestCookieJar object.

However, the management of cookies is automated when you use a Session object.

This means that you won’t have to send the cookies explicitly:

Copied! 
import requests session = requests.Session() response = session.get( 'https://httpbin.org/cookies', cookies='my-cookie': 'my-value'> ) # # "cookies": # "my-cookie": "my-value" # > # > print(response.text) 

Because it will be done for your automatically.

# Saving the requests cookies in a file and restoring them

You can also save the requests cookies in a file.

Copied! 
import json import requests response = requests.get('http://google.com', timeout=30) with open('cookies.txt', 'w', encoding='utf-8') as f: json.dump( requests.utils.dict_from_cookiejar(response.cookies), f ) 

The cookies.txt file stores the following JSON string.

Copied! 
"AEC": "Ad49MVEu4N64Tk1gEROw417s9FgcqdqeIeVZ8eL9m-HQldzOrLAF2HvxHQ"> 

Notice that we used the requests.utils.dict_from_cookiejar method to create a dictionary from the RequestCookieJar object.

We then passed the dictionary and the file object to the json.dump method.

The json.dump method serializes the supplied object as a JSON formatted stream and writes it to a file.

You can then read and restore the cookies from the file.

Copied! 
import json import requests session = requests.session() with open('cookies.txt', 'r', encoding='utf-8') as f: cookies = requests.utils.cookiejar_from_dict(json.load(f)) session.cookies.update(cookies) # print(session.cookies.get_dict()) 

We created a brand new Session object but you can use an existing Session .

1. We used the json.load() method to convert the contents of the file to a native Python dictionary.
2. We used the requests.utils.cookiejar_from_dict method to create a RequestCookieJar object from the dictionary.
3. The last step is to update the cookies of the Session object with the newly read cookies.

# Additional Resources

You can learn more about the related topics by checking out the following tutorials:

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.

Источник

How to send cookies in a post request with the Python Requests library?

I’m trying to use the Requests library to send cookies with a post request, but I’m not sure how to actually set up the cookies based on its documentation. The script is for use on Wikipedia, and the cookie(s) that need to be sent are of this form:

enwiki_session=17ab96bd8ffbe8ca58a78657a918558e; path=/; domain=.wikipedia.com; HttpOnly 

cookies = dict(cookies_are='working') 

How can I encode a cookie like the above using this library? Do I need to make it with python’s standard cookie library, then send it along with the POST request?

Your cookie consists of a number of a=b; pairs. At a guess, use a as the key and b as the value in a dictionary.

3 Answers 3

The latest release of Requests will build CookieJars for you from simple dictionaries.

import requests cookies = r = requests.post('http://wikipedia.org', cookies=cookies) 

@ThiefMaster: Normally, cookies live in browsers. If this answer doesn’t have to do with browsers, what does it have to do with?

@ChrisNielsen: This question and the code in the answer is about setting a cookie in a Python request. The request does something similar to a browser request, but no browsers are involved.

@ChrisNielsen : cookies are an aspect of the basic http protocol, which is most often used by browsers: datatracker.ietf.org/doc/html/rfc6265

Just to extend on the previous answer, if you are linking two requests together and want to send the cookies returned from the first one to the second one (for example, maintaining a session alive across requests) you can do:

import requests r1 = requests.post('http://www.yourapp.com/login') r2 = requests.post('http://www.yourapp.com/somepage',cookies=r1.cookies) 

Additionally, you can use requests.session for this exact thing, storing cookies across multiple sessions, making calls from the returned session object instead.

I’ve had to utilize this even when using sessions at times. Sessions seem to miss Set-Cookie headers in some situations.

@kervin this just happened to me as well. Seems like a bug in requests, because session should handle that.

If you want to pass the cookie to the browser, you have to append to the headers to be sent back. If you’re using wsgi:

import requests . def application(environ, start_response): cookie = response_headers = [('Content-type', 'text/plain')] response_headers.append(('Set-Cookie',cookie)) . return [bytes(post_env),response_headers] 

I’m successfully able to authenticate with Bugzilla and TWiki hosted on the same domain my python wsgi script is running by passing auth user/password to my python script and pass the cookies to the browser. This allows me to open the Bugzilla and TWiki pages in the same browser and be authenticated. I’m trying to do the same with SuiteCRM but i’m having trouble with SuiteCRM accepting the session cookies obtained from the python script even though it has successfully authenticated.

Источник