PHP Streams
Note: This post is over two years old and so the information contained here might be out of date. If you do spot something please leave a comment and we will endeavour to correct.
Streams are a way of generalising file, network, compression resources and a few other things in a way that allows them to share a common set of features. I stream is a resource object that has streamable behaviour. It can be read or written to in a linear fashion, but not necessarily from the beginning of the stream.
Streams have been available in PHP for quite a while (at least since version 4.3.0) and are used pretty transparently by most PHP programmers. They can be used to access files, network resources, command line arguments, pretty much anything that goes through the input/output stream in PHP.
I was recently looking at ReactPHP and found that the use of streams was a requirement in order to prevent blocking the input/output stream. Although, I had seen streams being used in PHP applications, I wasn’t entirely certain how to use them myself. As a result I thought I’d put together a post about them.
Streams come in especially handy when accessing large files. Let’s say we needed to access a log file that was a few hundred megabytes in size. We could just read it the file into memory with file_get_contents() and then run through the contents until we find what we need. The problem with this approach is that we would quickly run out of system resources and the program would just error before we had a chance to access the data.
A much better solution to this is to use PHP streams. We can use the function fopen() to get a file handle and then stream the file a little bit at a time, thus reducing the amount of memory needed to read the file.
This is the transparent stream usage I mentioned above. Whilst we haven’t explicitly used any stream functions or syntax we are still using PHP streams to read in chunks of the file.
Stream Syntax
You can access streams directly using the syntax [scheme]://[target]. The scheme is the name of the wrapper and the target is what is being read or written, which largely depends on the thing being read. PHP has a number of different built in wrappers, transports and stream filters. You can find out what they are using the following three functions.
On my current system this outputs this.
tcp [1] => udp [2] => unix [3] => udg [4] => ssl [5] => tls [6] => tlsv1.0 [7] => tlsv1.1 [8] => tlsv1.2 ) Array ( [0] => https [1] => ftps [2] => compress.zlib [3] => compress.bzip2 [4] => php [5] => file [6] => glob [7] => data [8] => http [9] => ftp [10] => phar [11] => zip ) Array ( [0] => zlib.* [1] => bzip2.* [2] => convert.iconv.* [3] => string.rot13 [4] => string.toupper [5] => string.tolower [6] => string.strip_tags [7] => convert.* [8] => consumed [9] => dechunk ) This gives plenty of options to connect to all sorts of streams.
The php:// Stream
The built in php:// stream is perhaps the most versatile stream wrapper and a good starting point to see how to use streams. An example of the php:// wrapper being used is with the stdin target. This is a read only stream that will read the input from the standard input and create it as a stream in PHP. We can then read the contents of the stream through the fgets() function. This reads one line in at a time, but is fine for what we are doing. We could also use fread() to do this as well.
This script can be used by piping the output of a file into the script on the command line.
The output of the script is the contents of the file printed onto the command line, but the core idea of piping into a PHP script is outlined here.
Another example is using php://input to read the input to the script. This is another read only stream that can be used to read the raw body of POST requests to the script. Let’s say we called a script with the following curl request.
We can access the POST data by using the $_POST super-global variable.
This will print out the following.
thing [data2] => anotherthing ) By using php://input we can read out the contents of the POST data directly as an input stream.
This outputs the following.
PHP 5.1 introduced php://memory and php://temp which allow reading and writing to either memory or a temporary file.
The php://memory can be used in much the same way as the php://input stream, but in this case we can both read and write to it. Here is an example of opening a memory stream and then writing some data to it.
No file was created during this code, we are just writing to memory. We can also read from the memory stream as we normally would with files and streams. Note that you can’t do this straight away though, we must first rewind the pointer to the start of the stream before we can read the contents of the stream. This can be done using the rewind() function.
The php://temp stream acts like php://memory, but the crucial difference is that if you write more than a certain amount of data (by default 2MB) then a file is created to store this information. The location of this file depends on your system, but you can find this information out by using the sys_get_temp_dir() function. This file will be automatically deleted when the PHP script ends.
It is also possible to control how much data you can add to the php://temp stream before a file is created. This is done by appending «/maxmemory:n», where ‘n’ is the maximum amount of data to keep in memory before using a temporary file, in bytes.
With that in place, if the data we write exceeds more than 1028 bytes then a file is created in the temporary directory.
Streaming A Web Page
You might have seen that http and https are available as stream wrappers. This means that we can open up a web page as a stream resource and stream it in as data. Here is an example of this in action.
This returns the following output.
Note that we are cutting off the end of the html tag, but this is because we are only reading in the first 50 bytes of data, which is where the data stops.
Filters
Filters are a form of meta-stream that allow you to manipulate data as it passes through a stream. This means you can add filters to function calls like readfile() or fopen() without having to change data before (or after) it passes through the stream. The syntax of filters is php://filter/read=[filter]/resource=[stream] with the filter being from stream_get_filters() above and the stream being any usable stream.
For example, we can expand on the php://stdin example above by passing the input through a filter. In this case we are changing all of the text to be uppercase.
Another example is to swap out the resource part with a web address. This will open the web address as a stream.
When we read the contents of the filtered stream it will be in uppercase.
We can do the opposite of this by using the file_put_contents() with a similar filter setup. The snippet below will write the text ‘hashbangcode’, in uppercase, to a file called uppertext.txt. Note the use of the write filter instead of the read filter.
Stream Contexts
Another powerful part of streams is creating contexts. This allows us to change the way in which the streams are used. For example, when we use the file_get_contents() function to access a URL it will always do this using a GET request.
In order to change the type of request being made we create a stream context and pass this to the function. Stream context are created using the stream_context_create() function, which takes an array that details the context that needs to be created.
The following is an example of creating the needed context to create a POST request (including the content of the request) using stream_context_create() and then using file_get_contents(). I’m using https://postman-echo.com/post/ to test this post request as it will return whatever you sent it and so is a good way of testing that everything worked.
'hello world', ]; $context = stream_context_create( [ 'http' => [ 'method' => 'POST', 'header' => [ 'Accept: application/json', 'Content-Type: application/x-www-form-urlencoded' ], 'content' => http_build_query($data), ], ] ); echo file_get_contents("https://postman-echo.com/post/", null, $context); Running this returns the following.
,"headers":,"json":,"url":"https://postman-echo.com/post/">%Another good example of using stream contexts is when bypassing the strict SSL rules that PHP has. Let’s say we were trying to connect to a local server that doesn’t have a fully signed SSL certificate. We can create a stream context that allows the SSL verification step to be bypassed.
[ 'verify_peer' => false, 'verify_depth' => 5, 'allow_self_signed' => true, 'verify_peer_name' => false, ] ] ); echo file_get_contents("https://localhost", null, $context);
Conclusion
There are a lot more to streams in PHP than what I have detailed here. PHP includes a bunch of functions that allow you to interact with streams in lots of different ways. It is also possible to create custom stream wrappers and filters, which I might detail in later posts.
tmpfile
Creates a temporary file with a unique name in read-write-binary (w+b) mode and returns a file handle.
The file is automatically removed when closed (for example, by calling fclose() , or when there are no remaining references to the file handle returned by tmpfile() ), or when the script ends.
If the script terminates unexpectedly, the temporary file may not be deleted.
Parameters
This function has no parameters.
Return Values
Returns a file handle, similar to the one returned by fopen() , for the new file or false on failure.
Examples
Example #1 tmpfile() example
$temp = tmpfile ();
fwrite ( $temp , «writing to tempfile» );
fseek ( $temp , 0 );
echo fread ( $temp , 1024 );
fclose ( $temp ); // this removes the file
?>
The above example will output:
See Also
- tempnam() — Create file with unique file name
- sys_get_temp_dir() — Returns directory path used for temporary files
User Contributed Notes 7 notes
To get the underlying file path of a tmpfile file pointer:
$file = tmpfile ();
$path = stream_get_meta_data ( $file )[ ‘uri’ ]; // eg: /tmp/phpFx0513a
I found this function useful when uploading a file through FTP. One of the files I was uploading was input from a textarea on the previous page, so really there was no «file» to upload, this solved the problem nicely:
# Upload setup.inc
$fSetup = tmpfile ();
fwrite ( $fSetup , $setup );
fseek ( $fSetup , 0 );
if (! ftp_fput ( $ftp , «inc/setup.inc» , $fSetup , FTP_ASCII )) echo «
Setup file NOT inserted
» ;
>
fclose ( $fSetup );
?>
The $setup variable is the contents of the textarea.
And I’m not sure if you need the fseek($temp,0); in there either, just leave it unless you know it doesn’t effect it.
Since this function may not be working in some environments, here is a simple workaround:
function temporaryFile($name, $content)
$file = DIRECTORY_SEPARATOR .
trim(sys_get_temp_dir(), DIRECTORY_SEPARATOR) .
DIRECTORY_SEPARATOR .
ltrim($name, DIRECTORY_SEPARATOR);
register_shutdown_function(function() use($file) unlink($file);
>);
at least on Windows 10 with php 7.3.7, and Debian Linux with php 7.4.2,
the mode is not (as the documentation states) ‘w+’ , it is ‘w+b’
(an important distinction when working on Windows systems)
To get tmpfile contents:
$tmpfile = tmpfile ();
$tmpfile_path = stream_get_meta_data ( $tmpfile )[ ‘uri’ ];
// . write to tmpfile .
$tmpfile_content = file_get_contents ( $tmpfile_path );
?>
Perhaps not the best way for production code, but good enough for logging or a quick var_dump() debug run.
No, the fseek() is necessary — after writing to the file, the file pointer (I’ll use «file pointer» to refer to the current position in the file, the thing you change with fseek()) is at the end of the file, and reading at the end of the file gives you EOF right away, which manifests itself as an empty upload.
Where you might be getting confused is in some systems’ requirement that one seek or flush between reading and writing the same file. fflush() satisfies that prerequisite, but it doesn’t do anything about the file pointer, and in this case the file pointer needs moving.
Beware that PHP’s tmpfile is not an equivalent of unix’ tmpfile.
PHP (at least v. 5.3.17/linux I’m using now) creates a file in /tmp with prefix «php», and deletes that file on fclose or script termination.
So, if you want to be sure that you don’t leave garbage even in case of a fatal error, or killed process, you shouldn’t rely on this function.
Use the classical method of deleting the file after creation:
$fn = tempnam ( ‘/tmp’ , ‘some-prefix-‘ );
if ( $fn )
$f = fopen ( $fn , ‘w+’ );
unlink ( $fn ); // even if fopen failed, because tempnam created the file
if ( $f )
do_something_with_file_handle ( $f );
>
>
?>