php-fpm: Avoiding «Cannot allocate memory» Error with better configuration
My php-fpm failed and I wonder why. My server does not have so much memory. Did php crash because it wanted to spawn more processes but could because there was no memory? Or did it crash because it could not spawn more processes (which I do not think)?
[11-Aug-2014 08:17:57] NOTICE: fpm is running, pid 3385 [11-Aug-2014 08:17:57] NOTICE: ready to handle connections [11-Aug-2014 14:50:38] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:40] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:41] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:42] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 16 total children [11-Aug-2014 14:50:43] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:45] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 16 total children [11-Aug-2014 14:50:45] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:49] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 16 total children [11-Aug-2014 14:50:49] ERROR: fork() failed: Cannot allocate memory (12) [11-Aug-2014 14:50:51] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 16 total childrenWith the php RAM limit of 128M and 16 «total children», that means that php tried to allocate 16*128M = 2G of memory? Or are only the 8 children who php wants to «spawn» are used at the initial phase of failure? Then php tried to allocate 0,5G of RAM and failed.
I changed the configuration as follow and I hope it will be more stable:
sed -i 's|pm.start_servers = 5|pm.start_servers = 4|' /etc/php-fpm-5.5.d/www.conf sed -i 's|pm.max_children = 50|pm.max_children = 4|' /etc/php-fpm-5.5.d/www.conf sed -i 's|pm.min_spare_servers = 5|pm.min_spare_servers = 4|' /etc/php-fpm-5.5.d/www.conf sed -i 's|pm.max_spare_servers = 35|pm.max_spare_servers = 4|' /etc/php-fpm-5.5.d/www.confI found this to determine the limit setting of pm.max_children. Is it correct that this setting along with the setting for the maximum of RAM used by php [standard is 128 MB] limits all php-fpm usage of RAM? I wonder why php tries to spawn more and more children.
— the following command will help us to determine the memory used by each (PHP-FPM) child process:
ps -ylC php-fpm —sort:rss
The RSS column shows non-swapped physical memory usage by PHP-FPM processes in kilo Bytes.
On an average each PHP-FPM process took ~75MB of RAM on my machine.
Appropriate value for pm.max_children can be calculated as:
pm.max_children = Total RAM dedicated to the web server / Max child process size – in my case it was 85MB
The server has 8GB of RAM, so:
pm.max_children = 6144MB / 85MB = 72
And I get this from php itself: «pm.min_spare_servers and pm.max_spare_servers cannot be greater than pm.max_children»
Composer proc_open(): fork failed — Cannot allocate memory
OK so it was CloudLinux limiting the memory for the user to 1024mb, because it works when the limit is doubled to 2048mb.
That’s the same setting on our previous server (CentOS/CloudLinux 6.x) but it looks like each version of CentOS is much more memory hungry than the rest.
Whats weird is that running composer with —profile shows the most it uses is 482mb. Even if it doubles when forking (as has been suggested) that’s still below the 1024mb limit.
Solution 2
I ran into the same problem. My system had 1.5GB RAM free and it was not enough. Composer was eating too much memory very fast.
My only solution was to clear cache and update to latest version (1.4.2):
composer clear-cache sudo composer selfupdate Related videos on Youtube
Pdiggity
Updated on September 16, 2022
Comments
The following exception is caused by a lack of memory and not having swap configured Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:974 Stack trace: 0 [internal function]: Composer\Util\ErrorHandler::handle(2, 'proc_open(): fo. ', 'phar:///home/te. ', 974, Array) 1 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(974): proc_open('stty -a | grep . ', Array, NULL, NULL, NULL, Array) 2 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(784): Symfony\Component\Console\Application->getSttyColumns() 3 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(745): Symfony\Component\Console\Application->getTerminalDimensions() 4 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(675): Symfony\Component\Console\Application->getTerminalWidth() 5 phar:///home/tea/composer in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php on line 974 . but with a large instance: 4gb RAM and 4gb swap. The free RAM is never exhausted, let alone the available/cached RAM, and the swap isn’t touched!
total used free shared buff/cache available Mem: 3788 885 1908 9 993 2692 Swap: 3967 0 3967 It’s the first time running composer update on this new machine, CentOS/CloudLinux 7.1 (with cPanel). In desperation, I’ve tried # php -dmemory_limit=1G ../composer.phar update —no-scripts —prefer-dist and I’ve tried removing the composer.lock and vendor files and even tried adding a temporary swap file but it really doesn’t seem to be a memory problem — could the error be misguided? proc_open is not disabled and I also tried with shell fork bomb protection disabled but no jive. Would love a heads up. N.B. I’m aware of the advice to commit the composer.lock file and do a composer install but this instance is being used for dev (as was the previous CentOS/CloudLinux 6.x machine with smaller resource specs) so we need to use the same methods we were using previously.
Composer proc_open(): fork failed — Cannot allocate memory
OK so it was CloudLinux limiting the memory for the user to 1024mb, because it works when the limit is doubled to 2048mb.
That’s the same setting on our previous server (CentOS/CloudLinux 6.x) but it looks like each version of CentOS is much more memory hungry than the rest.
Whats weird is that running composer with —profile shows the most it uses is 482mb. Even if it doubles when forking (as has been suggested) that’s still below the 1024mb limit.
Solution 2
I ran into the same problem. My system had 1.5GB RAM free and it was not enough. Composer was eating too much memory very fast.
My only solution was to clear cache and update to latest version (1.4.2):
composer clear-cache sudo composer selfupdate Related videos on Youtube
Pdiggity
Updated on September 16, 2022
Comments
The following exception is caused by a lack of memory and not having swap configured Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:974 Stack trace: 0 [internal function]: Composer\Util\ErrorHandler::handle(2, 'proc_open(): fo. ', 'phar:///home/te. ', 974, Array) 1 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(974): proc_open('stty -a | grep . ', Array, NULL, NULL, NULL, Array) 2 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(784): Symfony\Component\Console\Application->getSttyColumns() 3 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(745): Symfony\Component\Console\Application->getTerminalDimensions() 4 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(675): Symfony\Component\Console\Application->getTerminalWidth() 5 phar:///home/tea/composer in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php on line 974 . but with a large instance: 4gb RAM and 4gb swap. The free RAM is never exhausted, let alone the available/cached RAM, and the swap isn’t touched!
total used free shared buff/cache available Mem: 3788 885 1908 9 993 2692 Swap: 3967 0 3967 It’s the first time running composer update on this new machine, CentOS/CloudLinux 7.1 (with cPanel). In desperation, I’ve tried # php -dmemory_limit=1G ../composer.phar update —no-scripts —prefer-dist and I’ve tried removing the composer.lock and vendor files and even tried adding a temporary swap file but it really doesn’t seem to be a memory problem — could the error be misguided? proc_open is not disabled and I also tried with shell fork bomb protection disabled but no jive. Would love a heads up. N.B. I’m aware of the advice to commit the composer.lock file and do a composer install but this instance is being used for dev (as was the previous CentOS/CloudLinux 6.x machine with smaller resource specs) so we need to use the same methods we were using previously.