Curl php follow redirect

How to Follow Redirects with cURL for CLI or PHP

Let’s take a really common example. Say we want to follow redirects with cURL for google.com . It is common to curl follow redirect a URL.

Follow Redirects with cURL in the CommandLine#

If you navigate directly to google in the command-line using curl , you will be taken to a 301 redirect URL.

This makes sure that you follow the proper canonical URL as set by the particular website in question.

We can easily curl follow redirect to the end redirection by adding in the -L flag.

Follow Redirects with cURL in PHP#

If you’re doing this in PHP and want to find out where the redirected page goes, instead of actually going there yourself, you can do something like this to curl follow redirect:

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $result = curl_exec($ch); if (preg_match('~Location: (.*)~i', $result, $match))

This is particularly good for following URLs shorteners , such as the likes of Bit.ly and TinyURL to name only two.

Top Related Articles

1. How to use Docker to test any Linux distribution locally
2. How to monitor Network Traffic on Linux
3. How to use Screen on Linux
4. How to build a website quickly using PHP
5. Just SSH to my Jump Box!
6. Create daterange array of missing dates
7. Speedtest on Ubuntu Server (commandline)
8. URL GET vars to PHP Array
9. Remove specific HTML tags using PHP
10. Let Joomla and MySQL interact!

Источник

Following redirects with Curl in PHP.

As a good web citizen, I try to always follow redirects. Not just in my browser, where I actually don’t have all that much control over things, but also a consumer of web services.

When doing requests with CURL, redirects are not followed by default.

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_POSTFIELDS, "foo"); curl_setopt($curl, CURLOPT_POST, true); curl_exec($curl); ?> 

Assuming the given url actually redirects like this:

HTTP/1.1 301 Moved Permanently Location: /newendpoint 

Curl will automatically just stop. To make it follow redirects, the FOLLOWLOCATION setting is needed, as such:

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_POSTFIELDS, "foo"); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_POST, true); curl_exec($curl); ?> 

CURLOPT_FOLLOWLOCATION will follow the redirects up to 5 times (by default).

However, if you look at the second request, it actually does a GET request after the POST .

This is also the default behavior for browsers, but actually non-conforming with the HTTP standard, and also not desirable for consumers of web services.

To fix this, all you have to do is use CURLOPT_CUSTOMREQUEST instead of CURLOPT_POST :

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_POSTFIELDS, "foo"); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST"); curl_exec($curl); ?> 

Streams

After doing this, the secondary request will be a POST request as well. There’s one more issue though, if you were doing a POST or a PUT request you probably had a request body attached.

There’s two ways to supply a request body, as a string or as a stream. If we were uploading a file it makes much more sense to use a stream, because it unlike posting a string, a stream doesn’t have to be kept in memory.

To upload a stream with curl, you need CURLOPT_PUT and CURLOPT_INFILE . Don’t let the name CURLOPT_PUT fool you, it’s use for every request, and without CURLOPT_PUT , CURLOPT_INFILE is ignored.

For example, this is how we could upload a large file using POST.

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_PUT, true); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($curl, CURLOPT_INFILE, fopen('largefile.json', 'r')); curl_exec($curl); ?> 

This will work great, unless the target location redirects. If it does, curl will throw the following error:

Necessary data rewind wasn't possible (code #65) 

This seems to be related to PHP bug #47204.

Basically this means that you cannot use CURLOPT_INFILE and CURLOPT_FOLLOWLOCATION together. There’s two alternatives:

1. Don’t use CURLOPT_INFILE , but send the request body as a string instead, with CURLOPT_POSTFIELDS .
2. Don’t use CURLOPT_FOLLOWLOCATION , but instead manually check if the response was a 3xx redirect and manually follow each hop.

Strings

Using CURLOPT_POSTFIELDS you can supply a request body as a string. Lets try to upload our earlier failed request using that method:

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($curl, CURLOPT_POSTFIELDS, file_get_contents('largefile.json')); curl_exec($curl); ?> 

This also will not work exactly as you expect. While the second request to /someredirect will still be a POST request, it will be sent with an empty request body.

To fix this, use the undocumented CURLOPT_POSTREDIR option.

 $curl = curl_init('http://example.org/someredirect'); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($curl, CURLOPT_POSTFIELDS, file_get_contents('largefile.json')); curl_setopt($curl, CURLOPT_POSTREDIR, 3); curl_exec($curl); ?> 

According to the PHP changelog, this was added in PHP 5.3.2, and according to PHP bug #49571 there are four possible values:

0 -> do not set any behavior 1 -> follow redirect with the same type of request only for 301 redirects. 2 -> follow redirect with the same type of request only for 302 redirects. 3 -> follow redirect with the same type of request both for 301 and 302 redirects. 

Looking for a CTO or senior developer for your next project? I’m looking for contracts or full-time gigs! Check out my resume or drop me a line! —>

Web mentions

Comments

MeadSteve • Jul 23, 2013 I guess when you get a 301/308 it’d normally be worth logging something as well, for the maintainer of the code to take some action. Versus a 302/307 where you’d be happy for the code to do this silently.

Tom Binga • Mar 20, 2014 You helped solved a problem that’s been holding me back for a week or so now. Thank you!

Marcos Saturno • Jun 03, 2014 Very good info! I’m still having problems when trying to write onto web HDFS (hadoop) from a PHP script. If I make a PUT request passing CREATE as a parameter, it will provide me a 307 temporary redirect, so i’d need to make a second PUT request to the new URL provided. I’m not able to follow it or at least slipt the redirect URL from the HTTP Response. Could you please help me with it? DOC:
http://hadoop.apache.org/do. I’m using something like: $options = array( // CURLOPT_PUT => true, CURLOPT_HEADER => true, CURLOPT_CUSTOMREQUEST => «PUT», CURLOPT_FOLLOWLOCATION => true ); $ch = curl_init(‘:/webhdfs/v1/user/USER?op=CREATE’); // Execute
curl_setopt_array($ch, $options); curl_exec($ch); //echo curl_errno($ch); if(!curl_errno($ch))
< $info = curl_getinfo($ch); echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url']; >// Close handle curl_close($ch); ?>

Evert • Jun 03, 2014 Does your request not have a body? I wonder if that messes things up. You may want to PUT an empty string instead of nothing at all, because curl may fall back to ‘GET’ behavior (although with a PUT method).

Marcos Saturno • Jun 03, 2014 Hmm, good tip! But actually I found some classes ready to use WEB HDFS:
https://github.com/simpleen. Tx for the help, anyways! Regards, Marcos.

Roger Qiu • Jun 08, 2014 I don’t feel secure knowing my request might be redirected to some place I don’t know.

Ajai • Mar 19, 2015 i am doing an request using Advanced REST Client and i find some redirects (not cached) , but i get the actual response. But when i try the same using PHP curl it doesnt work , i dont get the output , but the redirected output is displayed. Please can any one help me

Hassan Nomani Alvi • Feb 27, 2016 I am trying to post data using curl.After posting I would like to go to the url.In other words I am trying to get same functionality as we get with form method=»post» and action=»someurl.php» .How to do this?Thanks in advance.

Источник

PHP: Following redirects with cURL.

This is a short guide on how to force PHP’s cURL functions to follow a HTTP redirect. By default, cURL will not follow any redirect headers that are sent by the server.

301 Moved.

If you send a cURL request that doesn’t follow redirects, you may end receiving the following response from the server:

301 Moved. The document has moved here.

Note that you might also receive a 302 header. This all depends on the server and the URL that you are sending a request to.

Take the following PHP example:

//The URL of the page that we want to retrieve. $url = 'http://google.com'; //Initiate cURL and pass it the URL we want to retrieve. $ch = curl_init($url); //Tell cURL to return the output as a string. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); //Execute the cURL request and return the output to a string. $response = curl_exec($ch); //Print out the response to the browser. echo $response;

The above code will output “301 Moved” response because Google has a 301 redirect on the URL in question and our cURL client has not been configured to follow it.

PHP and the CURLOPT_FOLLOWLOCATION option.

This is where the CURLOPT_FOLLOWLOCATION option comes in useful. This option tells the cURL client that it should automatically follow any redirects specified in the “Location:” header. Hence the name “follow location”.

In PHP, you can use this option like so:

//Tell cURL that it should follow any redirects. curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

If you add the line above to our original code, you will see that our cURL client now follows Google’s 301 redirect.

Using the CURLOPT_MAXREDIRS option.

If your cURL client follows redirects, then it is a good idea to also use the CURLOPT_MAXREDIRS option. This allows us to specify the maximum number of redirects that we should follow before stopping. If we do not set this option, then a simple server misconfiguration could kill our client by sending it too many redirect headers. The CURLOPT_FOLLOWLOCATION option is recursive, which means that you could get stuck in an endless redirect loop.

//Tell cURL that it should only follow 3 redirects at most. curl_setopt($ch, CURLOPT_MAXREDIRS, 3);

The PHP code above tells cURL that it should only follow 3 redirects. If a 4th redirect is encountered, a “CURLE_TOO_MANY_REDIRECTS” error will be thrown.

Источник

CURL — если сервер отдает редирект

Бывает так что сервер перенаправляет на другой URL. Например Google, если перейти на https://google.com c IP из РФ он делает 302-й редирект на https://www.google.ru . Чтобы CURL сам переходил на новый URL, нужно добавить параметр:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

function curl_redir_exec($ch) < static $curl_loops = 0; static $curl_max_loops = 20; if ($curl_loops >= $curl_max_loops) < $curl_loops = 0; return false; >curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $data = curl_exec($ch); $dd = explode("\r\n\r\n", $data); // или $dd = explode("\r\n", $data); $header = $dd[0]; $data = @$dd[1]; $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE); if ($http_code == 301 || $http_code == 302) < $matches = array(); preg_match('/Location:(.*?)\n/', $header, $matches); $url = @parse_url(trim(array_pop($matches))); if (!$url) < $curl_loops = 0; return $data; >$last_url = parse_url(curl_getinfo($ch, CURLINFO_EFFECTIVE_URL)); if (empty($url['scheme'])) < $url['scheme'] = $last_url['scheme']; >if (empty($url['host'])) < $url['host'] = $last_url['host']; >if (empty($url['path'])) < $url['path'] = $last_url['path']; >$new_url = $url['scheme'] . '://' . $url['host'] . $url['path'] . ($url['query'] ? '?' . $url['query'] : ''); curl_setopt($ch, CURLOPT_URL, $new_url); return curl_redir_exec($ch); > else < $curl_loops = 0; return $data; >> $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'https://google.com'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_HEADER, false); $html = curl_redir_exec($ch); curl_close($ch); echo $html;

Источник