Large arrays in php

array_chunk

Chunks an array into arrays with length elements. The last chunk may contain less than length elements.

Parameters

When set to true keys will be preserved. Default is false which will reindex the chunk numerically

Return Values

Returns a multidimensional numerically indexed array, starting with zero, with each dimension containing length elements.

Errors/Exceptions

If length is less than 1 , a ValueError will be thrown.

Changelog

Version	Description
8.0.0	If length is less than 1 , a ValueError will be thrown now; previously, an error of level E_WARNING has been raised instead, and the function returned null .

Examples

Example #1 array_chunk() example

$input_array = array( ‘a’ , ‘b’ , ‘c’ , ‘d’ , ‘e’ );
print_r ( array_chunk ( $input_array , 2 ));
print_r ( array_chunk ( $input_array , 2 , true ));
?>

The above example will output:

Array ( [0] => Array ( [0] => a [1] => b ) [1] => Array ( [0] => c [1] => d ) [2] => Array ( [0] => e ) ) Array ( [0] => Array ( [0] => a [1] => b ) [1] => Array ( [2] => c [3] => d ) [2] => Array ( [4] => e ) )

See Also

User Contributed Notes 20 notes

Tried to use an example below (#56022) for array_chunk_fixed that would «partition» or divide an array into a desired number of split lists — a useful procedure for «chunking» up objects or text items into columns, or partitioning any type of data resource. However, there seems to be a flaw with array_chunk_fixed — for instance, try it with a nine item list and with four partitions. It results in 3 entries with 3 items, then a blank array.

So, here is the output of my own dabbling on the matter:

function partition ( $list , $p ) $listlen = count ( $list );
$partlen = floor ( $listlen / $p );
$partrem = $listlen % $p ;
$partition = array();
$mark = 0 ;
for ( $px = 0 ; $px < $p ; $px ++) $incr = ( $px < $partrem ) ? $partlen + 1 : $partlen ;
$partition [ $px ] = array_slice ( $list , $mark , $incr );
$mark += $incr ;
>
return $partition ;
>

$citylist = array( «Black Canyon City» , «Chandler» , «Flagstaff» , «Gilbert» , «Glendale» , «Globe» , «Mesa» , «Miami» ,
«Phoenix» , «Peoria» , «Prescott» , «Scottsdale» , «Sun City» , «Surprise» , «Tempe» , «Tucson» , «Wickenburg» );
print_r ( partition ( $citylist , 3 ) );

?>

Array
(
[0] => Array
(
[0] => Black Canyon City
[1] => Chandler
[2] => Flagstaff
[3] => Gilbert
[4] => Glendale
[5] => Globe
)

[1] => Array
(
[0] => Mesa
[1] => Miami
[2] => Phoenix
[3] => Peoria
[4] => Prescott
[5] => Scottsdale
) [2] => Array
(
[0] => Sun City
[1] => Surprise
[2] => Tempe
[3] => Tucson
[4] => Wickenburg
)

Источник

Accessing Big Arrays in PHP

I’ve been doing some profiling on different methods of accessing large(ish) arrays of data in PHP. The use case is pretty simple: some of our tools output data into PHP files as associative arrays and these files are considered static data by the application. We make games, so some examples of data files would include items in a catalog, tasks that a user must complete, or definitions for maps:

Since these arrays are large-ish(400K), and much of our code is interested in this data, it becomes necessary to access this data as efficiently as possible. I settled on timing 3 different patterns for doing this. After presenting the methods I will share my results below.

What I’m looking for is some experience based validation on these methods and their timing as well as any other methods to try out.

Method #1: getter function

In the method, the exporter actually creates a file that looks like:

Client code can then get the data by simply calling getSomeData() when they want it.

Method #2: global + include

In this method the data file looks identical to the original code block above, however the client code must jump through a few hoops to get the data into a local scope. This assumes the array is in a file called ‘some_data.php’;

global $some_data; //must be the same name as the variable in the data file. include 'some_data.php'; Copy code

This will bring the $some_data array into scope, though it is a bit cumbersome for client code (my opinion).

Method #3: getter by reference

This method is nearly identical to Method #1, however the getter function does not return a value but rather sets a reference to the data.

Client code then retrieves the data by declaring a local variable (called anything) and passing it by reference to the getter:

$some_data_anyname = array(); getSomeDataByRef(&$some_data_anyname); Copy code

Results

So I ran a little script that runs each of these methods of retrieving data 1000 times on and averages the run time (computed by microtime(true) at the beginning and end). The following are my results (in ms, running on a MacBookPro 2GHz, 8GB RAM, PHP version 5.3.4):

METHOD #1:

AVG: 0.0031637034416199 MAX: 0.0043289661407471 MIN: 0.0025908946990967

METHOD #2:

AVG: 0.01434082698822 MAX: 0.018275022506714 MIN: 0.012722969055176

METHOD #3:

AVG: 0.00335768699646 MAX: 0.0043489933013916 MIN: 0.0029017925262451

It seems pretty clear, from this data anyway, that the global+include method is inferior to the other two, which are «negligible» difference.

Thoughts? Am I completely missing anything? (probably. )

Answer

Solution:

Not sure if this is exactly what your looking for but it should help out with speed and memory issues. You can use the fixed spl array:

$startMemory = memory_get_usage(); $array = new SplFixedArray(100000); for ($i = 0; $i < 100000; ++$i) < $array[$i] = $i; >echo memory_get_usage() - $startMemory, ' bytes'; Copy code

Also have you thought about storing the data in a cache/memory? For example you could use mysqlite with the inmemory engine on the first execution then access data from there:

$pdo = new PDO('sqlite::memory:'); $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // .. Use PDO as normal Copy code

Answer

Solution:

For one of my projects in which a database was not an option, I faced the same problem of loading big (by big I mean series of 3 MB files) php files containing arrays in memory and I was looking for options to maximize the performances. I found a very easy one which was caching these files on the disk as json at first use. I divided load time by 3 and also memory peak consumption by 30%. Loading local json file with json_decode() is much much faster than including a big php file containing an array. it also has the advantage of being a format that most languages can manipulate directly. Hope that helps.

Share solution ↓

Additional Information:

Didn’t find the answer?

Our community is visited by hundreds of web development professionals every day. Ask your question and get a quick answer for free.

Similar questions

Find the answer in similar questions on our website.

Write quick answer

Do you know the answer to this question? Write a quick response to it. With your help, we will make our community stronger.

About the technologies asked in this question

PHP

PHP (from the English Hypertext Preprocessor — hypertext preprocessor) is a scripting programming language for developing web applications. Supported by most hosting providers, it is one of the most popular tools for creating dynamic websites. The PHP scripting language has gained wide popularity due to its processing speed, simplicity, cross-platform, functionality and distribution of source codes under its own license.
https://www.php.net/

MySQL

DBMS is a database management system. It is designed to change, search, add and delete information in the database. There are many DBMSs designed for similar purposes with different features. One of the most popular is MySQL. It is a software tool designed to work with relational SQL databases. It is easy to learn even for site owners who are not professional programmers or administrators. MySQL DBMS also allows you to export and import data, which is convenient when moving large amounts of information.
https://www.mysql.com/

HTML

HTML (English «hyper text markup language» — hypertext markup language) is a special markup language that is used to create sites on the Internet. Browsers understand html perfectly and can interpret it in an understandable way. In general, any page on the site is html-code, which the browser translates into a user-friendly form. By the way, the code of any page is available to everyone.
https://www.w3.org/html/

Welcome to programmierfrage.com

programmierfrage.com is a question and answer site for professional web developers, programming enthusiasts and website builders. Site created and operated by the community. Together with you, we create a free library of detailed answers to any question on programming, web development, website creation and website administration.

Get answers to specific questions

Ask about the real problem you are facing. Describe in detail what you are doing and what you want to achieve.

Help Others Solve Their Issues

Our goal is to create a strong community in which everyone will support each other. If you find a question and know the answer to it, help others with your knowledge.

Источник