Requests python session set cookies

How To Get / Set Http Headers, Cookies And Manage Sessions Use Python Requests Module

In the previous article How To Use Python Requests Module To Send Get Or Post Request Example, we have learned how to install and use the python requests module to send HTTP Get and Post request to a web server. It also tells you how to post form data or pass query string parameters use the python requests module. This article will tell you how to use the python requests module to manage HTTP headers, cookies, and sessions.

1. Get / Set HTTP Headers Use Python Requests Module.

1.1 Get Server Response HTTP Headers.

Python requests module’s headers property is used to get HTTP headers. The headers property is a dictionary-type object, you should provide the header name to get the header value.

>>> import requests >>> response = requests.get("http://www.dev2qa.com") >>> response.headers['content-type'] 'text/html; charset=UTF-8'

If you want to get all headers, just call response.headers , it will list out all HTTP response headers in a JSON format string.

1.2 Add Custom Headers To HTTP Request.

import requests # Create a python dictionary object to save custom http headers. custom_header = # Pass above custom_header dictionary object to requests module's get function's headers parameter. response = requests.get('https://www.dev2qa.com', headers=custom_header)

2. Get / Set Cookies Use Python Requests Module.

2.1 Python Get HTTP Cookies.

1. Get all HTTP response cookies by invoke response.cookies property. This property is an instance of requests.cookies.RequestsCookieJar class.

>>> import requests >>> response = requests.get(‘http://www.dev2qa.com’) >>> response.cookies

>>> for cookie in response.cookies: . print('cookie domain = ' + cookie.domain) . print('cookie name = ' + cookie.name) . print('cookie value = ' + cookie.value) . print('*************************************') . cookie domain = .dev2qa.com cookie name = __cfduid cookie value = d8dccd301af7c19f3286442254fe10aa51560824472 ************************************* cookie domain = .dev2qa.com cookie name = active_template::80464 cookie value = pub_site.1560824472 ************************************* .

>>> for item in rsponse.cookies.items(): . print(item) . ('__cfduid', 'd0c1cc559af0893eb31cd23bb9bc22a261560866202') ('active_template::80464', 'pub_site.1560866203') ('ezCMPCCS', 'true') ('ezepvv', '92') ('ezoab_80464', 'mod31') ('ezoadgid_80464', '-1') ('ezopvc_80464', '1') ('ezoref_80464', '') ('ezovid_80464', '1774096901') ('ezovuuid_80464', '2d40f1eb-b006-4540-75cb-09a37a3b8387') ('ezovuuidtime_80464', '1560866206') ('lp_80464', 'https://www.dev2qa.com/?user_name=jerry&password=jerry') >>> for item in cookies.iterkeys(): . print(item) . __cfduid active_template::80464 ezCMPCCS ezepvv ezoab_80464 ezoadgid_80464 ezopvc_80464 ezoref_80464 ezovid_80464 ezovuuid_80464 ezovuuidtime_80464 lp_80464

# Get cookie value by provided cookie name. >>> response.cookies['ezCMPCCS'] 'true'

# Use get_dict function to get related domain's cookies. >>> response.cookies.get_dict('.dev2qa.com') 

>>> response.cookies.list_domains() ['.dev2qa.com']

2.2 Python Send HTTP Cookies To Web Server.

# Import python requests module. >>> import requests >>> # Set url value. >>> url = 'https://www.dev2qa.com' >>> # Create a dictionary object. >>> cookies = dict(name='jerry', password='888') >>> # Use python requests module to get related url and send cookies to it with cookies parameter. >>> response = requests.get(url, cookies=cookies)

>>> import requests >>> >>> url = 'https://www.dev2qa.com' >>> # Create a RequestsCookieJar object. >>> cookies_jar = requests.cookies.RequestsCookieJar() >>> # Add first cookie, the parameters are cookie_key, cookie_value, cookie_domain, cookie_path. >>> cookies_jar.set('name', 'jerry', domain='dev2qa.com', path='/cookies') Cookie(version=0, name='name', value='jerry', port=None, port_specified=False, domain='dev2qa.com', domain_specified=True, domain_initial_dot=False, path='/cookies', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest=, rfc2109=False) >>> # Add second cookie. >>> cookies_jar.set('password', 'jerry888', domain='dev2qa.com', path='/cookies') Cookie(version=0, name='password', value='jerry888', port=None, port_specified=False, domain='dev2qa.com', domain_specified=True, domain_initial_dot=False, path='/cookies', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest=, rfc2109=False) >>> # Get url with cookie parameter. >>> response = requests.get(url, cookies=cookies_jar)

3. Use Session In Python Requests Module.

Python requests module’s session() method will return a request.sessions.Session object, then all the later operations ( such as to browse a related URL page ) on this session object will use one same session.

>>> import requests >>> # Call requests module’s session() method to return a requests.sessions.Session object. >>> session = requests.session() >>> >>> print(session)

The returned request.sessions.Session objects provide a lot of attributes and methods for you to get related headers, cookie value in the same session. The session object also provides a get method to request a web page by URL. Below is an example of how to use the python requests module session object.

# Show all headers and cookies in this session. >>> session.headers >>> >>> session.cookies >>> # Use this session object to get a web page by url. >>> response = session.get('http://www.dev2qa.com') >>> # When above browse web page process complete, this session has cookies. >>> session.cookies >> >>> custom_header = # Get web page url and send custom headers. >>> >>> response = session.get('http://www.dev2qa.com', headers=custom_header)

The below link will show you python requests module session class source code, you can read it if you need it.

Источник

Requests python session set cookies

Last updated: Jul 2, 2023
Reading time · 4 min

# Table of Contents

# Python: How to get and set Cookies when using Requests

Use the Session class to set and get cookies when using the requests module in Python.

The class creates a Session object that stores the cookies and all requests that are made handle cookies automatically.

Copied! 
import requests session = requests.Session() print(session.cookies.get_dict()) # <> response = session.get('http://google.com') # print(session.cookies.get_dict()) 

Make sure you have the requests module installed to be able to run the code sample.

Copied! 
pip install requests # or with pip3 pip3 install requests 

The Session object enables you to persist cookies across requests.

The object persists cookies across all requests that were made using the Session instance.

The Session object has all of the methods of the main requests API.

The get_dict() method returns a Python dictionary of the name-value pairs of cookies.

You can also use Sessions as context managers.

Copied! 
import requests with requests.Session() as session: print(session.cookies.get_dict()) # <> response = session.get('http://google.com') # print(session.cookies.get_dict()) 

Make sure the code that accesses the Session object is inside the indented block.

# Accessing the path and the domain when using Cookies with requests

It is very likely, that you will also have to access the domain and the path to which the request was made when accessing the cookies.

You can use a list comprehension to construct a list of dictionaries that contain the path and domain.

Copied! 
import requests session = requests.Session() print(session.cookies.get_dict()) # <> print('-' * 50) response = session.get('http://google.com') # print(session.cookies.get_dict()) print('-' * 50) result = [ 'name': c.name, 'value': c.value, 'domain': c.domain, 'path': c.path> for c in session.cookies ] # [] print(result) 

We used a list comprehension to iterate over the RequestCookieJar object ( session.cookies ) and returned a dictionary on each iteration.

The dictionary contains the name and value of the cookie, the path and the domain.

# Sending cookies with a request

If you want to send cookies with a request, set the cookies keyword argument.

Copied! 
import requests session = requests.Session() response = session.get( 'https://httpbin.org/cookies', cookies='my-cookie': 'my-value'> ) # # "cookies": # "my-cookie": "my-value" # > # > print(response.text) 

We set the cookies keyword argument to a dictionary of key-value pairs.

# Accessing the cookies attribute directly on the Response object

In more recent versions of the requests module, you can also access the cookies attribute directly on the Response object.

Copied! 
import requests response = requests.get('http://google.com', timeout=30) # print(response.cookies.get_dict()) result = [ 'name': c.name, 'value': c.value, 'domain': c.domain, 'path': c.path> for c in response.cookies ] # [] print(result) 

Notice that we didn’t instantiate the Session class.

We simply accessed the cookies attribute on the Response object and called the get_dict() method on the RequestCookieJar object.

However, the management of cookies is automated when you use a Session object.

This means that you won’t have to send the cookies explicitly:

Copied! 
import requests session = requests.Session() response = session.get( 'https://httpbin.org/cookies', cookies='my-cookie': 'my-value'> ) # # "cookies": # "my-cookie": "my-value" # > # > print(response.text) 

Because it will be done for your automatically.

# Saving the requests cookies in a file and restoring them

You can also save the requests cookies in a file.

Copied! 
import json import requests response = requests.get('http://google.com', timeout=30) with open('cookies.txt', 'w', encoding='utf-8') as f: json.dump( requests.utils.dict_from_cookiejar(response.cookies), f ) 

The cookies.txt file stores the following JSON string.

Copied! 
"AEC": "Ad49MVEu4N64Tk1gEROw417s9FgcqdqeIeVZ8eL9m-HQldzOrLAF2HvxHQ"> 

Notice that we used the requests.utils.dict_from_cookiejar method to create a dictionary from the RequestCookieJar object.

We then passed the dictionary and the file object to the json.dump method.

The json.dump method serializes the supplied object as a JSON formatted stream and writes it to a file.

You can then read and restore the cookies from the file.

Copied! 
import json import requests session = requests.session() with open('cookies.txt', 'r', encoding='utf-8') as f: cookies = requests.utils.cookiejar_from_dict(json.load(f)) session.cookies.update(cookies) # print(session.cookies.get_dict()) 

We created a brand new Session object but you can use an existing Session .

1. We used the json.load() method to convert the contents of the file to a native Python dictionary.
2. We used the requests.utils.cookiejar_from_dict method to create a RequestCookieJar object from the dictionary.
3. The last step is to update the cookies of the Session object with the newly read cookies.

# Additional Resources

You can learn more about the related topics by checking out the following tutorials:

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.

Источник