Python speed up loops

Speeding up Python Code: Fast Filtering and Slow Loops

List comprehensions, boolean indexing and just-in-time (JIT) compilation for up to 200x speed up.

W hen exploring a new dataset and wanting to do some quick checks or calculations, one is tempted to lazily write code without giving much thought about optimization. While this might be useful in the beginning, it can easily happen that the time waiting for code execution overcomes the time that it would have taken to write everything properly.

This article shows some basic ways on how to speed up computation time in Python. With the example of filtering data, we will discuss several approaches using pure Python, numpy, numba, pandas as well as k-d-trees.

Fast Filtering of Datasets

As an example task, we will tackle the problem of efficiently filtering datasets. For this, we will use points in a two-dimensional space, but this could be anything in an n-dimensional space, whether this is customer data or the measurements of an experiment.

Let’s suppose we would like to extract all the points that are in a rectangle with between [0.2, 0.4] and [0.4, 0.6]. The naive way to do this would be to loop for each point and to check whether it fulfills this criterion. Codewise, this could look like as follows: First, we create a function to randomly distribute points in n-dimensional space with numpy, then a function to loop over the entries. To measure computation time we use timeit and visualize the filtering results using matplotlib.

Loop: 72 ms ± 2.11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

As we can see, for the tested machine it took approx. 70 ms to extract the points within a rectangle from a dataset of 100.000 points.

Источник

The Art of Speeding Up Python Loop

There is no “best” looping technique in Python, only the most suitable

“W hat is the fastest looping technique in Python?”

It’s a valid question, but often over-simplified. If there is a single looping method that is league above the rest, then the other techniques would have been deprecated.

As with most things in life, there will be situations where one significantly outperforms the others, and in some other cases, it’s absolute garbage. They’re situational. A better way to think of looping optimisation in Python should look like this.

Each of them are useful in their own right. Rather than creating yet another speed test article, I’d like to highlight what makes them unique, when to use them, and how to make them better? You’ll be surprised that with a simple syntax tweak, we can improve their performance by up to 8000 times faster.

Types of Looping Techniques:

• Loop Iteration
• List Comprehension
• Lambda Function
• Vectorised Operation

Loop Iteration

For-loops are the entry level looping technique taught to Python beginners because they are easy to read and extremely versatile.

items = ['bat', 'cat', 'dog']
for item in items: 
print(item)>> bat
>> cat
>> dog

However, many consider for-loops as the antithesis of efficient programming. They’re slow because Python’s implementation of for-loops have very heavy overheads (e.g: type checking, etc — to be discussed later) that is executed every time it iterates.

There are two main types of for-loops, namely (i) Index Loop and (ii) Row Loop.

Index Loop

An Index Loop takes a sequence of numbers (e.g: [0, 1, 2, …]) and runs your code logic for every element within the sequence.

Источник

A Super-Fast Way to Loop in Python

Do you think Python is slow? Here’s a fast way to loop in Python

Python is known for being a slow programming language. Although it’s a fact that Python is slower than other languages, there are some ways to speed up our Python code.

How? Simple, optimize your code.

If we write code that consumes little memory and storage, not only we’ll get the job done, but also make our Python code run faster.

Here’s a fast and also a super-fast way to loop in Python that I learned in one of the Python courses I took (we never stop learning!).

The average loop

Say we want to sum the numbers from 1 to 100000000 (we might never do that but that big number will help me make my point).

A typical approach would be to create a variable total_sum=0 , loop through a range and increment the value of total_sum by i on every iteration.

This gets the job done, but it takes around 6.58 seconds.

Although that doesn’t look so slow now, it’ll get slower as you add more 0’s to the number inside the range.

A faster way to loop using built-in functions

A faster way to loop in Python is using built-in functions.

In our example, we could replace the for loop with the sum function. This function will sum the values inside the range of numbers.

The code above takes 0.84 seconds. That’s way faster than the previous loop we used! This is why we should choose built-in functions over loops.

Источник