Python read entire file

How to read large text files in Python?

In this article, we will try to understand how to read a large text file using the fastest way, with less memory usage using Python.

To read large text files in Python, we can use the file object as an iterator to iterate over the file and perform the required task. Since the iterator just iterates over the entire file and does not require any additional data structure for data storage, the memory consumed is less comparatively. Also, the iterator does not perform expensive operations like appending hence it is time-efficient as well. Files are iterable in Python hence it is advisable to use iterators.

Problem with readline() method to read large text files

In Python, files are read by using the readlines() method. The readlines() method returns a list where each item of the list is a complete sentence in the file. This method is useful when the file size is small. Since readlines() method appends each line to the list and then returns the entire list it will be time-consuming if the file size is extremely large say in GB. Also, the list will consume a large chunk of the memory which can cause memory leakage if sufficient memory is unavailable.

Read large text files in Python using iterate

In this method, we will import fileinput module. The input() method of fileinput module can be used to read large files. This method takes a list of filenames and if no parameter is passed it accepts input from the stdin, and returns an iterator that returns individual lines from the text file being scanned.

Note: We will also use it to calculate the time taken to read the file using Python time.

Источник

Python Read Text file

Python provides built-in functions to perform file operations, such as creating, reading, and writing files. There are mainly two types of files that Python can handle, normal text files and binary files. In this tutorial, we will take a look at how to read text files in Python.

Steps to Read Text File in Python

In Python, to read a text file, you need to follow the below steps.

Step 1: The file needs to be opened for reading using the open() method and pass a file path to the function.

Step 2: The next step is to read the file, and this can be achieved using several built-in methods such as read() , readline() , readlines() .

Step 3: Once the read operation is performed, the text file must be closed using the close() function.

Now that we have seen the steps to read the file content let’s understand each of these methods before getting into examples.

Python open() function

The open() function opens the file if possible and returns the corresponding file object.

Syntax – open(file, mode=’r’, buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)

The open() function has a lot of parameters. Let’s take a look at the necessary params for reading the text file. It opens the file in a specified mode and returns a file object.

• file – path like object which represents the file path
• mode (Optional) – The mode is an optional parameter. It’s a string that specifies the mode in which you want to open the file.

Methods for Reading file contents

1. read() : The read() function returns the read bytes in the form of string. This method is useful when you have a small file, and you want to read the specified bytes or entire file and store it into a string variable.
2. readline() : The readline() function returns one line from a text file and retuns in the form of string.
3. readlines() : The readlines() function reads all the lines from the text file and returns each line as a string element in a list.

Python close() function

The file will remain open until you close the file using the close() function. It is a must and best practice to perform this operation after reading the data from the file as it frees up the memory space acquired by that file. Otherwise, it may cause an unhandled exception.

Examples for Reading a Text file in Python

Example 1 – Read the entire text file using the read() function

In the below example, we are reading the entire text file using the read() method. The file can be opened in the read mode or in a text mode to read the data, and it can be stored in the string variable.

# Program to read the entire file using read() function file = open("python.txt", "r") content = file.read() print(content) file.close() # Program to read the entire file (absolute path) using read() function file = open("C:/Projects/Tryouts/python.txt", "r") content = file.read() print(content) file.close()

Dear User, Welcome to Python Tutorial Have a great learning . Cheers

Example 2 – Read the specific length of characters in a text file using the read() function

There are times where you need to read the specific bytes in a file. In that case, you can use the read() function by specifying the bytes. The method will output only the specified bytes of characters in a file, as shown below.

# Program to read the specific length # of characters in a file using read() function file = open("python.txt", "r") content = file.read(20) print(content) file.close() 

Example 3 – Read a single line in a file using the readline() function

If you want to read a single line in a file, then you could achieve this using readline() function. You also use this method to retrieve specific bytes of characters in a line, similar to the read() method.

# Program to read single line in a file using readline() function file = open("python.txt", "r") content = file.readline() print(content) file.close()

Example 4- Read text file line by line using the readline() function

If you want to traverse the file line by line and output in any format, then you could use the while loop with the readline() method as shown below. This is the most effective way to read the text file line by line in Python.

# Program to read all the lines in a file using readline() function file = open("python.txt", "r") while True: content=file.readline() if not content: break print(content) file.close() 

Dear User, Welcome to Python Tutorial Have a great learning . Cheers

Example 5 – Read all the lines as a list in a file using the readlines() function

The readlines() method will read all the lines in the file and outputs in a list of strings, as shown below. Later you can use the list to traverse and extract the specified content from the list.

# Program to read all the lines as a list in a file # using readlines() function file = open("python.txt", "r") content=file.readlines() print(content) file.close() 

['Dear User,\n', 'Welcome to Python Tutorial\n', 'Have a great learning . \n', 'Cheers']

Источник