Variable CURL Response Time
I have a PHP CURL request to an HTTPS site which takes a JSON and request and returns a JSON response. Usually it should not take more than 1 second. But, I have seen that the CURL response time is variable. Sometimes, it takes 4 seconds to respond and sometimes 1.2 seconds. Why is this so and How can I make it faster and process the whole request in less than a second? Following is the curl response times for the same request in 3 different times..
Array ( [url] => https://xx.xx.xxx.xx/site/y [content_type] => application/json [http_code] => 200 [header_size] => 198 [request_size] => 358 [filetime] => -1 [ssl_verify_result] => 20 [redirect_count] => 0 [total_time] => 4.213941 [namelookup_time] => 2.1E-5 [connect_time] => 0.015284 [pretransfer_time] => 0.722955 [size_upload] => 222 [size_download] => 553 [speed_download] => 131 [speed_upload] => 52 [download_content_length] => 0 [upload_content_length] => 0 [starttransfer_time] => 4.015112 [redirect_time] => 0 ) Array ( [url] => https://xx.xx.xxx.xx/site/y [content_type] => application/json [http_code] => 200 [header_size] => 198 [request_size] => 358 [filetime] => -1 [ssl_verify_result] => 20 [redirect_count] => 0 [total_time] => 1.27581 [namelookup_time] => 8.3E-5 [connect_time] => 0.016223 [pretransfer_time] => 0.104996 [size_upload] => 222 [size_download] => 553 [speed_download] => 433 [speed_upload] => 174 [download_content_length] => 0 [upload_content_length] => 0 [starttransfer_time] => 1.273922 [redirect_time] => 0 ) Array ( [url] => https://xx.xx.xxx.xx/site/y [content_type] => application/json [http_code] => 200 [header_size] => 198 [request_size] => 358 [filetime] => -1 [ssl_verify_result] => 20 [redirect_count] => 0 [total_time] => 1.294147 [namelookup_time] => 7.0E-5 [connect_time] => 0.025723 [pretransfer_time] => 0.123456 [size_upload] => 222 [size_download] => 553 [speed_download] => 427 [speed_upload] => 171 [download_content_length] => 0 [upload_content_length] => 0 [starttransfer_time] => 1.292577 [redirect_time] => 0 ) curl_getinfo
// Check HTTP status code
if (! curl_errno ( $ch )) switch ( $http_code = curl_getinfo ( $ch , CURLINFO_HTTP_CODE )) case 200 : # OK
break;
default:
echo ‘Unexpected HTTP code: ‘ , $http_code , «\n» ;
>
>
// Close handle
curl_close ( $ch );
?>
Notes
Note:
Information gathered by this function is kept if the handle is re-used. This means that unless a statistic is overridden internally by this function, the previous info is returned.
User Contributed Notes 13 notes
Here are the response codes ready for pasting in an ini-style file. Can be used to provide more descriptive message, corresponding to ‘http_code’ index of the arrray returned by curl_getinfo().
These are taken from the W3 consortium HTTP/1.1: Status Code Definitions, found at
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
101=»Switching Protocols» [Successful 2xx]200=»OK»
201=»Created»
202=»Accepted»
203=»Non-Authoritative Information»
204=»No Content»
205=»Reset Content»
206=»Partial Content» [Redirection 3xx]300=»Multiple Choices»
301=»Moved Permanently»
302=»Found»
303=»See Other»
304=»Not Modified»
305=»Use Proxy»
306=»(Unused)»
307=»Temporary Redirect» [Client Error 4xx]400=»Bad Request»
401=»Unauthorized»
402=»Payment Required»
403=»Forbidden»
404=»Not Found»
405=»Method Not Allowed»
406=»Not Acceptable»
407=»Proxy Authentication Required»
408=»Request Timeout»
409=»Conflict»
410=»Gone»
411=»Length Required»
412=»Precondition Failed»
413=»Request Entity Too Large»
414=»Request-URI Too Long»
415=»Unsupported Media Type»
416=»Requested Range Not Satisfiable»
417=»Expectation Failed» [Server Error 5xx]500=»Internal Server Error»
501=»Not Implemented»
502=»Bad Gateway»
503=»Service Unavailable»
504=»Gateway Timeout»
505=»HTTP Version Not Supported»
And an example usage:
$ch = curl_init (); // create cURL handle (ch)
if (! $ch ) die( «Couldn’t initialize a cURL handle» );
>
// set some cURL options
$ret = curl_setopt ( $ch , CURLOPT_URL , «http://mail.yahoo.com» );
$ret = curl_setopt ( $ch , CURLOPT_HEADER , 1 );
$ret = curl_setopt ( $ch , CURLOPT_FOLLOWLOCATION , 1 );
$ret = curl_setopt ( $ch , CURLOPT_RETURNTRANSFER , 0 );
$ret = curl_setopt ( $ch , CURLOPT_TIMEOUT , 30 );
// execute
$ret = curl_exec ( $ch );
if (empty( $ret )) // some kind of an error happened
die( curl_error ( $ch ));
curl_close ( $ch ); // close cURL handler
> else $info = curl_getinfo ( $ch );
curl_close ( $ch ); // close cURL handler
if (empty( $info [ ‘http_code’ ])) die( «No HTTP code was returned» );
> else // load the HTTP codes
$http_codes = parse_ini_file ( «path/to/the/ini/file/I/pasted/above» );
// echo results
echo «The server responded:
» ;
echo $info [ ‘http_code’ ] . » » . $http_codes [ $info [ ‘http_code’ ]];
>
Setting Curl’s Timeout in PHP
I’m running a curl request on an eXist database through php. The dataset is very large, and as a result, the database consistently takes a long amount of time to return an XML response. To fix that, we set up a curl request, with what is supposed to be a long timeout.
$ch = curl_init(); $headers["Content-Length"] = strlen($postString); $headers["User-Agent"] = "Curl/1.0"; curl_setopt($ch, CURLOPT_URL, $requestUrl); curl_setopt($ch, CURLOPT_HEADER, false); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERPWD, 'admin:'); curl_setopt($ch,CURLOPT_TIMEOUT,1000); $response = curl_exec($ch); curl_close($ch); However, the curl request consistently ends before the request is completed (
7 Answers 7
CURLOPT_CONNECTTIMEOUT — The number of seconds to wait while trying to connect. Use 0 to wait indefinitely.
CURLOPT_TIMEOUT — The maximum number of seconds to allow cURL functions to execute.
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0); curl_setopt($ch, CURLOPT_TIMEOUT, 400); //timeout in seconds also don’t forget to enlarge time execution of php script self:
set_time_limit(0);// to infinity for example @PedroLobito what you mention is the default config of the php on cli, but it is possible that this may have been modified.
@cherouvim is obviously correct here (just run php -d max_execution_time=1 -r ‘while(true)’ or something to observe in action that the cli does not have a magic ‘always unlimited’ flag.
CURLOPT_CONNECTTIMEOUT = 0 doesn’t mean «indefinitely» curl.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html «Set to zero to switch to the default built-in connection timeout — 300 seconds.»
Hmm, it looks to me like CURLOPT_TIMEOUT defines the amount of time that any cURL function is allowed to take to execute. I think you should actually be looking at CURLOPT_CONNECTTIMEOUT instead, since that tells cURL the maximum amount of time to wait for the connection to complete.
While the docs in PHP say CURLOPT_TIMEOUT is about how long the function takes, the underlying curl library docs seem to say it’s about how long the request takes, which is an interesting distinction — not sure which way to read that!
There is a quirk with this that might be relevant for some people. From the PHP docs comments.
If you want cURL to timeout in less than one second, you can use CURLOPT_TIMEOUT_MS , although there is a bug/»feature» on «Unix-like systems» that causes libcurl to timeout immediately if the value is < 1000 ms with the error "cURL Error (28): Timeout was reached". The explanation for this behavior is:
«If libcurl is built to use the standard system name resolver, that portion of the transfer will still use full-second resolution for timeouts with a minimum timeout allowed of one second.»
What this means to PHP developers is «You can’t use this function without testing it first, because you can’t tell if libcurl is using the standard system name resolver (but you can be pretty sure it is)»
The problem is that on (Li|U)nix, when libcurl uses the standard name resolver, a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm.
The solution is to disable signals using CURLOPT_NOSIGNAL. Here’s an example script that requests itself causing a 10-second delay so you can test timeouts:
if (!isset($_GET['foo'])) < // Client $ch = curl_init('http://localhost/test/test_timeout.php?foo=bar'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_NOSIGNAL, 1); curl_setopt($ch, CURLOPT_TIMEOUT_MS, 200); $data = curl_exec($ch); $curl_errno = curl_errno($ch); $curl_error = curl_error($ch); curl_close($ch); if ($curl_errno >0) < echo "cURL Error ($curl_errno): $curl_error\n"; >else < echo "Data received: $data\n"; >> else < // Server sleep(10); echo "Done."; >PHP: Setting cURL timeout options.
This is a short guide on how to use the cURL timeout options in PHP. In certain cases, you may want to specify a timeout in order to prevent your HTTP request from taking too long.
There are two cURL timeout options that you need to know about. These options are slightly different from one another, so I will explain them now:
- CURLOPT_CONNECTTIMEOUT: The maximum amount of seconds that cURL should spend attempting to connect to a given URL.
- CURLOPT_TIMEOUT: The maximum amount of seconds it should take for all cURL operations to be carried out. i.e. The maximum amount of time that the request should take.
Take a look at the following piece of code.
In the code above, I set CURLOPT_CONNECTTIMEOUT to 10 seconds and I set CURLOPT_TIMEOUT to 30 seconds. This means:
- cURL should only spend 10 seconds attempting to connect to the given URL. If it can’t connect after 10 seconds, a timeout should occur.
- If it does connect successfully, it should only spend a maximum of 30 seconds executing the request. In other words, if we connect, but the server takes too long to respond in full, then cURL should call it a day and stop waiting.
Testing the cURL timeout.
To test this, I added some basic cURL error handling to the end of my script so that any cURL errors would result in an exception being thrown. The local URL that I am sending a HTTP request to uses PHP’s sleep function to create a 40 second delay (which is higher than the 30 seconds limit that we set with CURLOPT_TIMEOUT). As a result, the following exception was thrown:
Uncaught exception ‘Exception’ with message ‘Operation timed out after 30015 milliseconds with 0 bytes received’
As you can see, the cURL timeout after 30,015 milliseconds, which is 30 seconds.
Hopefully this tutorial cleared a few things up for you!