Php file line length

How Can I Read a Large File Line by Line in PHP?

Every now and then, you will have to read and process a large file in PHP. If you are not careful when reading a large file, you will often end up exhausting all the memory of your server. However, there is no need to worry because PHP already provides a lot of inbuilt functions to read large files (by large I mean files with size over 1GB) either line by line or one chunk at a time. In this article, you will learn about the pros and cons of all these techniques.

Reading a File Line by Line into an Array Using file()

You can use the file() function in PHP to read an entire file into an array. This function stores each line along with the newline character in its own array element. If you intended to read a file line by line and store the result in an array, the file() function seems to a natural choice.

One major drawback of this method is that it will read the whole file at once. In other words, it solves the problem of reading a file one line at a time but it still reads the whole file at once. This means that you won’t be able to use it to read very large files.

$life_of_bee_lines = file('life-of-the-bee.txt'); $short_lines = 0; // Output — Total Lines: 6305. echo "Total Lines: ".count($life_of_bee_lines).".\n"; foreach($life_of_bee_lines as $line) < if(strlen($line) > // Output — Total number of short lines: 778. echo "Total number of short lines: $short_lines.\n"; 

In the example above, I have read a text file called The Life of the Bee from Project Gutenberg. This file is just 360kb long so reading it using the file() function was not a problem. Now we will learn how to read a file that might be over 1GB in size without exhausting our memory.

Reading a Large File One Line at a Time Using fgets()

The fgets() function in PHP reads the current line from an open file pointed to by the resource handle. The function stops reading after reaching a specified length, encountering a new line or reaching the end of file. It also returns FALSE if there is no more data to be read. This means that we can check if a file has been read completely by comparing the return value of fgets() to false .

If you have not specified a particular length that this function should read, it will read the whole line. In other words, the maximum memory that you need with fgets() depends on the longest line in the file. Unless the file that you are reading has very long lines, you won’t exhaust your memory like you could with the file() function in previous section.

$handle = fopen("life-of-the-bee.txt", "r"); $total_lines = 0; $short_lines = 0; if ($handle) < $line = fgets($handle); while ($line !== false) < $total_lines++; if(strlen($line) $line = fgets($handle); > fclose($handle); > // Output — Total Lines: 6305. echo "Total Lines: $total_lines.\n"; // Output — Total number of short lines: 778. echo "Total number of short lines: $short_lines.\n"; 

Inside the if block, we read the first line of our file and if it is not strictly equal to false we enter the while loop and process the whole file one line at a time. We could also combine the file reading and checking process in one line like I have done the following example.

$handle = fopen("life-of-the-bee.txt", "r"); $total_lines = 0; $short_lines = 0; if ($handle) < while (($line = fgets($handle)) !== false) < $total_lines++; if(strlen($line) > fclose($handle); > // Output — Total Lines: 6305. echo "Total Lines: $total_lines.\n"; // Output — Total number of short lines: 778. echo "Total number of short lines: $short_lines.\n"; 

I would like to point out the just like file() , the fgets() function reads the new line character as well. You can verify this by using var_dump() .

Reading a Large File in Smaller Pieces Using fread()

One drawback of reading files using fgets() is that you will need substantial amount of memory to read files which have very long lines. The situation is very unlikely to happen but if you can’t make any assumptions about the file it is better to read it in small sized chunks.

This is where the fread() function in PHP comes to our rescue. This function reads a file up to length number of bytes. Youare the one who gets to specify the length so you don’t have to worry about running out of memory.

$handle = fopen("large-story.txt", "r"); $chunk_size = 1024*1024; $iterations = 0; if ($handle) < while (!feof($handle)) < $iterations++; $chunk = fread($handle, $chunk_size); >fclose($handle); > // Output — Read the whole file in 13 iterations. echo "Read the whole file in $iterations iterations."; 

In the above example, we read the large text file (1024*1024) 1MB at a time and it took 13 iterations to read the whole file. You can set the value of chunk_size to a reasonable value and read a very large file without exhausting your memory. You could use this technique to read a 1GB file using just 1MB of memory in about 1024 iterations. You can also read even larger files and increase the chunk_size if you want to keep the number of iterations low.

Quick Summary

Let’s recap everything that we have covered in this tutorial.

1. If you want to read files line by line into an array, using the file() function would be the right choice. The only drawback if that this function will read whole file at once so you won’t be able to read very large files without exhausting the memory.
2. If the files you are reading are very large and you have no plans to store the individual lines in an array, using the fgets() function would be the right choice. The amount of memory that your script needs will only depend on the length of largest line in the file that you are reading. This means that the file could be over 1GB in size but if the individual lines are not very large, you will never exhaust your memory.
3. If you are reading large files and have no idea how long individual lines could be in that particular file, using fread() would be your best bet. This function lets you specify the size of chunks in which you want to read the file. Another advantage of fread() over fgets() is that you will not have to perform a lot of iterations if the file you are reading contains large number of empty lines.

Let me know if there is anything that you would like me to clarify. Also, you are more than welcome to comment if you know other techniques for reading large files one line at a time in PHP.

Rate this post —

Источник

Read a Large File Line by Line in PHP

1. Use fgets() Function to Read a Large File Line by Line in PHP
2. Use file() Function to Read a Large File Line by Line in PHP
3. Use stream_get_line() Function to Read a Large File Line by Line in PHP

In this article, we will introduce methods to read a large file line by line in PHP.

Use fgets() Function to Read a Large File Line by Line in PHP

The built-in function fgets() reads a line from an open file. We can use this function to read a large file line by line. The correct syntax to use this function is as follows.

fgets($fileName, $lengthOfFile); 

The function fgets() has two parameters. The details of its parameters are as follows

This function returns a string of lengthOfFile -1 from the file specified by the user if successful. It returns False on failure.

The program below shows how we can use fgets() function to read a large file line by line. We will use fopen() and fclose() functions to open and close a file respectively. We have a text file named flowers.txt that contains the details of the flowers.

php if ($file = fopen("flowers.txt", "r"))   while(!feof($file))   $line = fgets($file);  echo ("$line");  >  fclose($file); > ?> 

We have used the feof() function to read the file until the end. The output shows the content of the file read line by line.

Use file() Function to Read a Large File Line by Line in PHP

In PHP, we can also use the file() function to read a file line by line. The file() function reads a file into an array line by line. The correct syntax to use this function is as follows:

file($fileName, $flags, $context) 

The program that reads the file using the file() function is as follows:

php foreach(file('flowers.txt') as $line)   echo $line. "\n"; > ?> 

Use stream_get_line() Function to Read a Large File Line by Line in PHP

We can also use the stream_get_line() function to read a file line by line. This function reads a file up to a delimiter. The correct syntax to use this function is as follows:

stream_get_line($handle, $length, $ending) 

The program that reads the file using stream_get_line() function is as follows:

php $file = fopen("flowers.txt", "r+"); while ($line = stream_get_line($file, 1024 * 1024, "."))   echo $line; > fclose($file); ?> 

Источник

PHP SplFileObject Standard Library

In this post, you will learn about the SplFileObject Standard Library of the PHP programming language.

PHP SplFileObject is a Standard PHP Library available from PHP version 5.1 (PHP 5 >= 5.1.0, PHP 7). It provides an Object Oriented Interface for file handling. It provides collection of interfaces and classes that are meant to solve common problems.

Advantages of SplFileObject over file_get_contents()

As we know, PHP also has a file_get_contents() method to deal with files. So, why and where should we use SplFileObject instead of file_get_contents(). The file_get_contents() method is not suitable for manipulating large files. It has some limited functionality when dealing with files. But SplFileObject provides an object-oriented approach to dealing with files. By using this, we can handle file manipulation by creating a class and extending SplFileObject. It contains several inbuilt methods and properties.

PHP splfileobject Example

?php $filename = "demo.txt"; $file = new SplFileObject($filename); ?>

In the above code, we have instantiated the SplFileObject.

Read File Contents using SplFileObject

?php $filename = "demo.txt"; $file = new SplFileObject($filename); foreach ($file as $line) < echo $line; >?>

The above example opens the current file and iterates over the content line by line.

PHP SplFileObject fread()

The fread() function is an inbuilt function of Standard PHP Library (SPL) which is used to read the given number of bytes from the file. Syntax —

string SplFileObject::fread( $length )

This function accepts single parameter $length which is used to specify the length to be read from file in bytes and returns the string read from the file on success or false on failure.

The given PHP program illustrate the SplFileObject::fread() function in PHP.

?php $filename = "demo.txt"; $file = new SplFileObject($filename); $gfg = $file->fread(5); ?>

PHP SplFileObject eof()

The eof() method is used to check the end of the file. It does not accept any parameter and always returns a Boolean value, i.e., either TRUE or FALSE. Syntax-

string SplFileObject::eof( void )

The given PHP program illustrate the SplFileObject::eof() function.

?php $filename = "demo.txt"; $file = new SplFileObject($filename); foreach ($file as $line) < while ( ! $file->eof()) < echo $line; >> ?>

PHP SplFileObject current()

The current() method is used to retrieve the current line of the file.

?php $filename = "demo.txt"; $file = new SplFileObject($filename); foreach ($file as $line) < echo ($file->key() + 1) . ': ' . $file->current(); > ?>

PHP SplFileObject fgets()

The SplFileObject::fgets() function is an inbuilt function of Standard PHP Library (SPL) in PHP which is used to get line number from the file. Syntax-

string SplFileObject::fgets( void )

This function does not accept any parameter. It returns an string containing the next line from the file return FALSE otherwise. The given PHP program illustrate the SplFileObject::fgets() function.

?php $filename = "demo.txt"; $file = new SplFileObject($filename); foreach ($file as $line) < echo $file->fgets(); > ?>

PHP SplFileObject getMaxLineLen()

The getMaxLineLen() method returns the maximum line length of the file if it is set in setMaxLineLen(), otherwise by default it returns 0.

?php $filename = "demo.txt"; $file = new SplFileObject($filename); var_dump($file->getMaxLineLen()); $file->setMaxLineLen(10); var_dump($file->getMaxLineLen()); ?>

Источник