Better determination of runnable CPU count #910
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will help to better determine the number of worker processes to create in certain situations. Unlike sysconf(_SC_NPROCESSORS_ONLN) this takes into account per-process cpu allowed masks as set by sched_setaffinity(2)/cpusets etc. So while a system may have 64 on-line cpu's, nginx itself may be limited to using just four of them in which case we should create just four worker processes not 64 under 'worker_processes auto;' sched_getaffinity(2) is available on at least Linux & FreeBSD.
With 'worker_processes auto;' the number of worker processes to create is generally determined at runtime via sysconf(_SC_NPROCESSORS_ONLN); However this value does not take into account any restrictions that have been put in place via mechanisms like cpuset(7)'s and/or sched_setaffinity(2). So for example while a system may have 18 "cpus" available as returned via sysconf(_SC_NPROCESSORS_ONLN) the actual number of allowed to run on cpus may be much lower. E.g. On a 18 core system, with 'worker_processes auto;' nginx will create 18 worker processes (+ the master). E.g. $ pgrep -c nginx 19 If however nginx has been restricted to run on just four. We will still create 18 worker processes. E.g. $ taskset -a -c 0-3 ./sbin/nginx -c conf/minimal.conf $ pgrep -c nginx 19 This can happen for example with 'docker --cpuset-cpus=', where you may end up with many more worker processes than runnable cpus. This commit uses sched_getaffinity() (where available, at least Linux & FreeBSD) to determine the number of actual available runnable cpus. So in the above example we now get $ taskset -a -c 0-3 ./sbin/nginx -c conf/minimal.conf $ pgrep -c nginx 5 Four worker processes plus the master. Closes: nginx#855
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull-request comprises two commits
The first adds a configure check for the sched_getaffinity(2) system
call.
The second makes use of sched_getaffinity(2) to better determine the
actual number of CPUs nginx is allowed to run on to better tune
worker_processes auto;.This number may be lower than that returned by
sysconf(_SC_NPROCESSORS_ONLN).This can be the case when running in containerised environments like
dockerwith--cpuset-cpus=set.This avoids ending up in a situation where you may end up with way more
worker_processes than actual runnable CPUs.