Open pickle file python

How to Read Pickle File Python?

Pickling data and storing it inside a file is a very useful concept and technique that actually helps the user to manage vast amounts of data. However, it not only helps in managing huge amounts of data but also helps the program load the data fast thus saving lots of crucial time.

This post is going to show the methods for reading data from a pickle file and displaying it on the terminal.

Pre-requisite: Create Pickle File

To read data from a pickle file, there must first exist a pickle file. Therefore, in this step, you are going to first create a new pickle file. If you already have an existing pickle file then you can skip this step. Use the following lines of code, to create a dictionary of a person in Python and store it inside a pickle file:

person1 = { «name» : «Marcus King» , «Age» : «22» , «Profession» : «Author» }

pickle . dump ( person1 , open ( «person1.p» , «wb» ) )

The above-mentioned code snippet will:

• Create a dictionary named “person1”.
• Create a new file named “person1.p”.
• Write the pickle data in the person1.p file.

Once the file has been created you can start working on the different methods or reading this pickle file data.

Method 1: Using pickle.load() method from Pickle Package

The pickle library contains a method which is the load() method which reads the pickled data from the file specified in its path. To demonstrate this start by importing the pickle library and opening up the file with read access using the following lines:

After that, use the load() method to load the pickled data and then store it inside a variable:

Print the data stored in the variable to verify that the pickle data has been read correctly:

The complete code for reading pickle data from the file created in the prerequisite step:

pickleFile = open ( «person1.p» , «rb» )

personInfo = pickle . load ( open ( «person1.p» , «rb» ) )
print ( personInfo )

Running this code will result in the following output on the terminal:

You have successfully read the data from a pickle file.

Method 2: Using read_pickle() method from Pandas Package

This method requires the use of the Pandas package as it contains the method “read_pickle()”. To use this, import the pandas library and open the pickle file with read access using the following lines:

pickleFile = open ( «person1.p» , «rb» )

Use the read_pickle() method and pass in the file as the argument and then store the result in an “obj” variable:

Complete code for this method is as:

pickleFile = open ( «person1.p» , «rb» )

obj = pd. read_pickle ( pickleFile )
print ( obj )

Executing this code will produce the following result on the terminal:

You have successfully read the pickle file using the Pandas library

Conclusion

To read the pickle file in Python, simply use the “load()” method from the pickle package or use the “read_pickle()” method from the pandas library. Both of these methods will require you to first open up the file using the open() method in the read access mode and then use the methods on them. This post explained the process of reading data from a pickle file.

About the author

Abdul Mannan

I am curious about technology and writing and exploring it is my passion. I am interested in learning new skills and improving my knowledge and I hold a bachelor’s degree in computer science.

Источник

How to Read Pickle file in Python : Various Methods with Step

Pickel in python is used to serialize and deserialize a Python object structure. Suppose you want to store data in a byte stream then you have to create a pickle file that will store all the information in a byte stream. Now you can store byte stream information in a file or database. In this tutorial, you will learn how to read a pickle file in python using various methods.

Steps to Read Pickel File in Python

In this section, you will learn how to read pickle files in python in steps. You have to follow all the steps defined here for a better understanding.

Step 1: Create a Dummy Pickle File

The first step is to create a sample pickle file that will be used for reading. However, if you have already saved the pickle file then you can move to the next steps.

Execute the below lines of code to create a dummy pickle file.

 df = pd.DataFrame(data=data) df.to_pickle("people.pkl")

Here I am first creating a sample dataframe that has some information and then saving it to a pickle file using the method df.to_pickle().

Dataframe contains the following information.

Step 2: Read Pickle File in Python

In this step you will know the various methods to read pickle file in python.

Method 1: Using the pickle module

In the first method, I will use the pickle module. Firstly I will open the pickle file and then appends its content with the empty list.

Run the below lines of code to implement this method.

Method 2: Read Pickle file in Python using Pandas package

The other method to read pickle file is using the pandas package. There is a read_pickle() function that allows you to read the file. The output will be dataframe.

Use the below lines of code to read the pickle file.

Output

Conclusion

A pickle file is a very useful file for storing data in the file or database. It is in byte stream and due to it, the space requirement is very less. In machine learning, if you have to build a model then you save the model in a pickle file.

These are the methods to read it. I hope you have liked this tutorial. If you have queries then you can contact us for more help.

Join our list

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

We respect your privacy and take protecting it seriously

Thank you for signup. A Confirmation Email has been sent to your Email Address.

Источник

Read a Pickle File Using Python

1. Read a Pickle File Using the pickle Module in Python
2. Read a Pickle File Using the pandas Module in Python

In Python, pickling refers to converting a Python object (lists, dictionaries, etc.) into a binary stream, and unpickling refers to converting a binary stream of data into a Python object.

The converted binary stream of data contains all the information to reconstruct the original object. Unfortunately, pickle files are generally considered unsafe.

Pickle files are used to save a program’s state (values of variables, objects, and their states, etc.), store Python objects to databases in the form of serialized binary strings, send data over TCP or Transmission Control Protocol, etc.

While training machine learning models, pickle files are used to store model weights, and sometimes, the loaded training data or the formatted training data is stored back to the disk in the form of pickle files.

In this article, we will get to understand how to read these pickle files using Python. We will discuss two such ways.

Read a Pickle File Using the pickle Module in Python

Python has an in-built module, pickle , that contains utilities for serializing and de-serializing data using Python. This data can be stored in pickle files.

We can use the pickle module to read a pickle file using Python. Refer to the following Python code for the same.

objects = [] file_name = "/path/to/the/pickle/file"  with (open(file_name, "rb")) as f:  while True:  try:  objects.append(pickle.load(f))  except EOFError:  break 

In the above code, the objects variable will hold all the data of the pickle file.

The code loops over the file to read it until an EOFError exception is found. The same is that the data is stored in objects inside a pickle file.

The load() function from the pickle module will only read a single object. After reading an object, the file pointer points to the beginning of the next object in the pickle file.

Refer to the documentation linked here to learn more.

Read a Pickle File Using the pandas Module in Python

We can use the pandas library to read a pickle file in Python.

The pandas module has a read_pickle() method that can be used to read a pickle file.

This method accepts a filepath_or_buffer argument: the file path, the URL, or the buffer from where the pickle file will be loaded. This function will return an unpickled object of the file.

Now let us see how to use this method practically. Refer to the following Python code for the same.

import pandas as pd  file_name = "/path/to/the/pickle/file" objects = pd.read_pickle(file_name) 

To learn more about the read_pickle() method, refer to the official documentation here.

Vaibhav is an artificial intelligence and cloud computing stan. He likes to build end-to-end full-stack web and mobile applications. Besides computer science and technology, he loves playing cricket and badminton, going on bike rides, and doodling.

Related Article — Python Pickle

Источник

Python Pickle Example

While we believe that this content benefits our community, we have not yet thoroughly reviewed it. If you have any suggestions for improvements, please let us know by clicking the “report an issue“ button at the bottom of the tutorial.

In this tutorial we will be discussing about Python Pickle Example. In our previous tutorial, we discussed about Python Multiprocessing.

Python Pickle

Python Pickle is used to serialize and deserialize a python object structure. Any object on python can be pickled so that it can be saved on disk. At first Python pickle serialize the object and then converts the object into a character stream so that this character stream contains all the information necessary to reconstruct the object in another python script. Note that the pickle module is not secure against erroneous or maliciously constructed data according to the documentation. So, never unpickle data received from an untrusted or unauthenticated source.

Python Pickle dump

In this section, we are going to learn, how to store data using Python pickle. To do so, we have to import the pickle module first. Then use pickle.dump() function to store the object data to the file. pickle.dump() function takes 3 arguments. The first argument is the object that you want to store. The second argument is the file object you get by opening the desired file in write-binary (wb) mode. And the third argument is the key-value argument. This argument defines the protocol. There are two type of protocol — pickle.HIGHEST_PROTOCOL and pickle.DEFAULT_PROTOCOL. See the sample code to know how to dump data using pickle.

import pickle # take user input to take the amount of data number_of_data = int(input('Enter the number of data : ')) data = [] # take input of the data for i in range(number_of_data): raw = input('Enter data '+str(i)+' : ') data.append(raw) # open a file, where you ant to store the data file = open('important', 'wb') # dump information to that file pickle.dump(data, file) # close the file file.close() 

The following program will prompt you to enter some input. In my case, it was like this.

Python Pickle load

To retrieve pickled data, the steps are quite simple. You have to use pickle.load() function to do that. The primary argument of pickle load function is the file object that you get by opening the file in read-binary (rb) mode. Simple! Isn’t it. Let’s write the code to retrieve data we pickled using the pickle dump code. See the following code for understanding.

import pickle # open a file, where you stored the pickled data file = open('important', 'rb') # dump information to that file data = pickle.load(file) # close the file file.close() print('Showing the pickled data:') cnt = 0 for item in data: print('The data ', cnt, ' is : ', item) cnt += 1 

Showing the pickled data: The data 0 is : 123 The data 1 is : abc The data 2 is : !@#$ 

Python Pickle Example

I made a short video showing execution of python pickle example programs — first to store data into file and then to load and print it. As you can see that the file created by python pickle dump is a binary file and shows garbage characters in the text editor.

Important Notes on Python Pickle

1. The pickle protocol is specific to Python — it’s not guaranteed to be cross-language compatible. This means you most likely can’t transfer the information to make it useful in other programming languages.
2. There is also no guarantee of compatibility between different versions of Python because not every Python data structure can be serialized by the module.
3. The latest version of the pickle protocol is used by default unless you manually change it.
4. Last but not least, the pickle module is not secure against erroneous or maliciously constructed data according to the documentation.

So, that’s all about python pickle example. Hope that you understand well. For any further query please use the comment section. 🙂 Reference: Official Documentation

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Источник