What is thread pool size in java

How to calculate thread pool size in Java

«How many thread to use in an executor pool: 22, 50, or 10?» — similar questions were asked many times when a thread pool is created, and we need to choose a number of threads in a pool. If we choose too big number of threads then it could end up with exhausting resources, CPU, memory. If we choose too small number of threads then throughput of a system will be poor as CPU/memory is not used enough. To choose properly number of threads for a threads pool we need to understand what type of tasks a thread pool will work with.
In general, all possible tasks can be divided into two groups:
compute-intensive tasks or CPU Bound tasks , I/O tasks or I/O Bound tasks .

Compute-intensive tasks (CPU bound)

Compute-intensive tasks in general are tasks when a CPU is constantly loaded with work and CPU idle time is much shorter comparing with usage time.

CPU | C | W | C | W | C | W | C. |-----------|---|-----------|---|-----------|---|--------> Time 

where C — compute time on CPU, W — idling time of CPU, waiting for a work. In case of compute-intensive tasks a system usually achieves optimum utilization with a thread pool size calculated by a formula below.

where N — number of CPU cores in a machine, which can obtained in Java with Runtime.getRuntime().availableProcessors() So, if a system has four CPU cores, then optimal maximum number of threads should be five.

I/O tasks (I/O Bound)

I/O tasks are tasks which involve a lot of communications with different applications on another a machine via network call. For example, when a thread runs a Http request to a remote system or fetches data from a database. I/O tasks are characterised with short processing time and longer idling time — CPU is less loaded with work and a thread waits for data to be obtained from a different machine.

CPU | C | W | C | W | C | W | C. |---|-------|---|---|------|---|--------> Time 

Threads count = N * U * (1 + W/C) 

1. a system has four CPU cores, so N = 4;
2. we target 100% CPU utilization, so U = 1;
3. we can estimate roughly for a request waiting time 100ms ( W ) and real calculating time is about 20ms ( C ).

Then we should expect that threads count should 20
as 4 * 1 * 100/20 = 20 .

Conclusion

This article describes in general an idea of how to calculate an optimal thread pool size. It would happen that thread pool size should be even greater than calculated one by a formula. Quite many aspects affect thread pool size, like if other applications are working on the same machine and how much CPU time that applications are consuming.

I would highlight that formulas give us an estimate size of a thread pool that should be verified in test and later in a production environment by observing CPU utilization.

Источник

What is optimum thread pool size for simple program running cpu based tasks in Java

Solution 1: If you have CPU bound tasks, as you increase the number of threads you get increasing overhead and slower performances. There is nothing wrong with ignoring advice that doesn’t apply to you, so you can and should increase the thread pool size.

What is optimum thread pool size for simple program running cpu based tasks in Java

Im using a thread pool to execute tasks , that are mostly cpu based with a bit of I/O, of size one larger than the number of cpus.

Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() + 1) 

Assuming case of simple program that submits all its tasks to this executor and does little else I assume having a thread pool any larger would slow things because the OS would have to timeslice it cpus more often chance to give each thread in the threadpool a chance to run.

Is that correct, and if so is this a real problem or mostly theoretical, i.e if I increased threadpool size to 1000 would I notice a massive difference.

If you have CPU bound tasks, as you increase the number of threads you get increasing overhead and slower performances. Note: having more threads than waiting tasks is just a waste of resources, but may not slow down the tasks so much.

I would use a multiple (e.g. 1 or 2) of the number of cpus rather than adding just one as having one too many threads can have a surprising amount of overhead.

For reference, check this description.

In short, what you have (No. CPU + 1) is optimal on average.

Java — What is the most performant way to setup, The objective of this question is to find the right choice of ThreadPool configuration to be used with Java7s CompletableFuture inside a Singleton …

Thread pool size should be much more than number of cores + 1

The recommended size for custom thread pools is number_of_cores + 1 (see here and here). So lets say there’s a Spring app on a system with 2 cores and configuration is something like this

In this case there will be one ExecutorService shared among several requests. So if 10 requests hit the server, only 3 of them can be simultaneously executed inside ExecutorService. This can create a bottleneck and the results will go worse with larger number of requests (remember: by default tomcat can handle up to 200 simultaneous requests = 200 threads). The app would perform much better without any pooling.

Usually one core can cope with more than one thread at a time. e.g. I created a service, which calls two times https://httpbin.org/delay/2. Each call takes 2 seconds to execute. So if no thread pool is used the service responds in average 4.5 seconds (tested this with 20 simultaneous requests). If thread pool is used the reponses vary depending on the pool size and hardware. I run a test on 4 core machine with different pool sizes. Below are the test results with min, max and average response times in milliseconds

From the results, it can be concluded that the best average time was with the max pool size. The average time with 5 (4 cores + 1) threads in the pool is even worse than the result without pooling. So in my opinion, if a request does not take much of a CPU time then it does not make sense to limit a thread pool to number of cores + 1 in web application.

Does anyone find anything wrong with setting a pool size to 20 (or even more) on 2 or 4 core machine for web services that are not CPU demanding?

Both of the pages you link to explicitly says that this only applies to unblocked, CPU bound tasks.

You have tasks that are blocked waiting on the remote machine, so this advice does not apply.

There is nothing wrong with ignoring advice that doesn’t apply to you, so you can and should increase the thread pool size.

Good advice comes with a rationale so you can tell when it becomes bad advice. If you don’t understanding why something should be done, then you’ve fallen into the trap of cargo cult programming, and you’ll keep doing it even when it’s no longer necessary or even becomes deleterious.

Raymond Chen, on his blog «The Old New Thing»

Java — Reasonable number of threads for thread pool, This stack size is configurable via -Xss (similar to -Xmx etc.). I believe the default is 512Kb per thread. At the moment I can’t find any authoritative to …

I’m planning to make a software with lot of peer to peer like network connections. Normally I would create an own thread for every connection to send and receive data, but in this case with 300-500+ connections it would mean continuously creating and destroying a lot of threads which would be a big overhead I guess. And making one thread that handles all the connections sequentially could probably slow down things a little. (I’m not really sure about this.)

The question is: how many threads would be optimal to handle this kind of problems? Would it be possible to calculate it in the software so it can decide itself to create less threads on an old computer with not as much resources and more on new ones?

It’s a theoretical question, I wouldn’t like to make it implementation or language dependant. However I think a lot of people would advice something like «Just use a ThreadPool , it will handle stuff like that» so let’s say it will not be a .NET application. (I’ll probably has to use some other parts of the code in an old Delphi project, so the language will be probably Delphi or maybe C++ but it’s not decided yet.)

Understanding the performance of your application under load is key, as mentioned before profiling, measurements and re-testing is the way to go.

As a general guide Goetz talks about having

number of CPUs * (1 + wait time / service time)

If this is Windows (you did mention .Net?), you should definitely implement this using I/O completion ports. This is the most efficient way to do Windows sockets I/O. There is an I/O-specific discussion of thread pool size at that documentation link.

The most important property of an I/O completion port to consider carefully is the concurrency value. The concurrency value of a completion port is specified when it is created with CreateIoCompletionPort via the NumberOfConcurrentThreads parameter. This value limits the number of runnable threads associated with the completion port. When the total number of runnable threads associated with the completion port reaches the concurrency value, the system blocks the execution of any subsequent threads associated with that completion port until the number of runnable threads drops below the concurrency value.

Basically, your reads and writes are all asynchronous and are serviced by a thread pool whose size you can modify. But try it with the default first.

A good, free example of how to do this is at the Free Framework. There are some gotchas that looking at working code could help you short-circuit.

You could do a calculation based on cpu speed, cores, and memory space in your install and set a constant somewhere to tell your application how many threads to use. Semaphores and thread pools come to mind.

Personally I would separate the listening sockets from the sending ones and open sending sockets in runtime instead of running them as daemons; listening sockets can run as daemons.

Multithreading can be its own headache and introduce many bugs. The best thing to do is make a thread do one thing and block when processing to avoid undesired and unpredictable results.

1. Make the number of threads configurable.
2. Target a few specific configurations that are the most common ones that you expect to support.
3. Get a good performance profiler / instrument your code and then rigorously test with different values of 1. for all the different types of 2. till you find an optimal value that works for each configuration.

I know, this might seem like a not-so smart way to do things but i think when it comes to performance, benchmarking the results via testing is the only sure-fire way to really know how well / badly it will work.

Edit: +1 to the question whose link is posted by paxDiablo above as a comment. Its almost the same question and theres loads of information there including a very detailed reply by paxDiablo himself.

Java — meaning of ‘core pool size’ in, Essentially, the corePoolSize is the number of Threads to maintain in the pool. eg. if you expect 10 concurrent requests on a regular basis, but peaks …

Источник