April 1, 2012 at 13:31Python threading or GIL is almost no hindrance
Probably everyone who has at least once been interested in Python knows about GIL - its both a strong and a weak point.
Without stopping single-threaded scripts from working, he puts a fair amount of sticks in the wheel during multi-threaded work on CPU-bound tasks (when threads are executed rather than hanging alternately waiting for I / O, etc.).
Details are well described in a two-year-old translation . We can’t overcome the GIL in the official Python assembly for true thread parallelization, but we can go the other way - prevent the system from transferring Python threads between the kernels. In general, a post from the series, “if you don’t need to, but really want to” :)
If you know about processor / cpu affinity, ctypes and pywin32 used, then there will be nothing new.
Let's take a simple code (almost like in a translation article):
Run on python 2.6.5 (ubuntu 10.04 x64, i5 750):
And on python 2.7.2 (win7 x64, i5 750):
Immediately discard that the win version is clearly slower. In both cases, a significant slowdown of the parallel version is visible.
GIL in any case will not allow the multi-threaded version to run faster than the linear one. However, if the implementation of a certain functionality is simplified by introducing threading into the code, then it is worth at least trying to reduce this gap as much as possible.
When a multi-threaded application is running, the OS can arbitrarily "transfer" different threads between the cores. And when two (or more) threads of the same python process simultaneously try to capture the GIL, the brakes begin. The transfer is also performed for a single-threaded program, but there it does not affect the speed.
Accordingly, in order for threads to capture the GIL one by one, you can limit the python process to one core. And the Affinity Mask CPU will help us in this, allowing in the format of bit flags to indicate which kernels / processors are allowedrun the program.
On different operating systems, this operation is performed by different means, but now we will consider Ubuntu Linux and WinXP +. FreeBSD 8.2 on Intel Xeon was also studied, but this will remain outside the scope of the article.
Before choosing kernels, you need to decide how many of them are at our disposal. Here it is worth dancing from the platform features: multiprocessing.cpu_count () in python 2.6+, os.sysconf ('SC_NPROCESSORS_ONLN') by POSIX, etc. An example definition can be found here .
To work directly with processor affinity, the following were selected:
To reach libc we will use the ctypes module . To load the desired library, we use ctypes.CDLL:
Everything would be fine, but there are two points:
Our functions:
pid_t is an int, cpu_set_t is a single-field structure of 1024 bits in size (i.e. it is possible to work with 1024 cores / processors).
We will use cpusetsize to work not immediately with all the kernels and assume that cpu_set_t is an unsigned long. In general, ctypes.Arrays should be used , but this is beyond the scope of the article.
It is also worth noting that mask is passed as a pointer, i.e. ctypes.POINTER (<type of the value itself>).
After matching the types C and ctypes we get:
After specifying argtypes, the types of values passed are monitored by ctypes. So that the module does not swear, but does its job, we will correctly indicate the values when called:
Apparently, ctypes itself implicitly dealt with the pointer. It is also worth noting that a call with pid = 0 is performed on the current process.
The documentation for the functions we need indicates:
Now we know when it will work and which library should be loaded.
We do it by analogy with the Linux version. Take the headlines:
As a HANDLE, ctypes.c_uint is quite suitable for us, but you need to be careful with the out parameter types:
DWORD_PTR is the same ctypes.c_uint, and P DWORD_PTR is ctypes.POINTER (ctypes.c_uint).
Total we get:
And it seems that we’ll do it there and it will work:
But alas . Functions accept not pid, but HANDLE of process. It still needs to be received. To do this, we use the OpenProcess function and the CloseHandle “paired” to it :
Without going into details, we just get the HANDLE of the process we need with access to read the parameters, and when ro = False, and to change them. This is written in the documentation for SetProcessAffinityMask and GetProcessAffinityMask:
So no Monte Carlo method :)
Rewrite our get_affinity and set_affinity to reflect the changes:
To slightly reduce the amount of code for the Win implementation, you can install the pywin32 module . It saves us from having to set constants and deal with libraries and call parameters. Our code above might look something like this:
Briefly, clearly, but this is a third-party module.
If you put it all together and add one more to our initial tests:
then the results will be as follows:
The numbers in the second column - cover those tests, but cnt 10 times Used on lshim.
We got two threads of execution with almost no loss in speed compared to the single-threaded version.
Affinity is set by a bit mask on both OSs. On a 4-core machine, get_affinity returns 15 (1 + 2 + 4 + 8).
An example and all the code for the article posted on github .
I accept any suggestions and complaints.
Also interested in the results on a processor with support for HT and other versions of Linux.
All from the first of April! This code really works :)
Without stopping single-threaded scripts from working, he puts a fair amount of sticks in the wheel during multi-threaded work on CPU-bound tasks (when threads are executed rather than hanging alternately waiting for I / O, etc.).
Details are well described in a two-year-old translation . We can’t overcome the GIL in the official Python assembly for true thread parallelization, but we can go the other way - prevent the system from transferring Python threads between the kernels. In general, a post from the series, “if you don’t need to, but really want to” :)
If you know about processor / cpu affinity, ctypes and pywin32 used, then there will be nothing new.
How it all started
Let's take a simple code (almost like in a translation article):
cnt = 100000000
trying = 2
def count():
n = cnt
while n>0:
n-=1
def test1():
count()
count()
def test2():
t1 = Thread(target=count)
t1.start()
t2 = Thread(target=count)
t2.start()
t1.join(); t2.join()
seq1 = timeit.timeit( 'test1()', 'from __main__ import test1', number=trying )/trying
print seq1
par1 = timeit.timeit( 'test2()', 'from __main__ import test2', number=trying )/trying
print par1
Run on python 2.6.5 (ubuntu 10.04 x64, i5 750):
10.41 13.25
And on python 2.7.2 (win7 x64, i5 750):
19.25 27.41
Immediately discard that the win version is clearly slower. In both cases, a significant slowdown of the parallel version is visible.
If you really want, then you can
GIL in any case will not allow the multi-threaded version to run faster than the linear one. However, if the implementation of a certain functionality is simplified by introducing threading into the code, then it is worth at least trying to reduce this gap as much as possible.
When a multi-threaded application is running, the OS can arbitrarily "transfer" different threads between the cores. And when two (or more) threads of the same python process simultaneously try to capture the GIL, the brakes begin. The transfer is also performed for a single-threaded program, but there it does not affect the speed.
Accordingly, in order for threads to capture the GIL one by one, you can limit the python process to one core. And the Affinity Mask CPU will help us in this, allowing in the format of bit flags to indicate which kernels / processors are allowedrun the program.
On different operating systems, this operation is performed by different means, but now we will consider Ubuntu Linux and WinXP +. FreeBSD 8.2 on Intel Xeon was also studied, but this will remain outside the scope of the article.
And how many options do we have?
Before choosing kernels, you need to decide how many of them are at our disposal. Here it is worth dancing from the platform features: multiprocessing.cpu_count () in python 2.6+, os.sysconf ('SC_NPROCESSORS_ONLN') by POSIX, etc. An example definition can be found here .
To work directly with processor affinity, the following were selected:
- Linux: sched_setaffinity / sched_getaffinity from libc
- Windows: SetProcessAffinityMask / GetProcessAffinityMask from kernel32
- POSIX as an offtopic: pthread_setaffinity_np / pthread_getaffinity_np
Linux Ubuntu
To reach libc we will use the ctypes module . To load the desired library, we use ctypes.CDLL:
libc = ctypes.CDLL( 'libc.so.6' )
libc.sched_setaffinity # наша функция
Everything would be fine, but there are two points:
- The hard name of libc.so.6 is not portable, and the libc.so file, which should be a symlink to the real version, is made a text file on Debian / Ubuntu.
At the moment, a crutch has been made in the form of a search for all files whose names begin with “libc.so” and an attempt to load them with OSError processing. Downloaded - this is our library.
If someone knows the best and universal solution - I will see it in the comments or in PM. - Specifying a function name is not enough. The number of parameters and their types are also needed. To do this, we use the “magic” attribute argtypes for the functions we need.
Our functions:
int sched_setaffinity(pid_t pid, size_t cpusetsize, cpu_set_t *mask);
int sched_getaffinity(pid_t pid, size_t cpusetsize, cpu_set_t *mask);
pid_t is an int, cpu_set_t is a single-field structure of 1024 bits in size (i.e. it is possible to work with 1024 cores / processors).
We will use cpusetsize to work not immediately with all the kernels and assume that cpu_set_t is an unsigned long. In general, ctypes.Arrays should be used , but this is beyond the scope of the article.
It is also worth noting that mask is passed as a pointer, i.e. ctypes.POINTER (<type of the value itself>).
After matching the types C and ctypes we get:
__setaffinity = _libc.sched_setaffinity
__setaffinity.argtypes = [ctypes.c_int, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t)]
__getaffinity = _libc.sched_getaffinity
__getaffinity.argtypes = [ctypes.c_int, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t)]
After specifying argtypes, the types of values passed are monitored by ctypes. So that the module does not swear, but does its job, we will correctly indicate the values when called:
def get_affinity(pid=0):
mask = ctypes.c_ulong(0) # инициализируем переменную
c_ulong_size = ctypes.sizeof(ctypes.c_ulong) # данные только по первым 32/64 ядрам
if __getaffinity(pid, c_ulong_size, mask) < 0:
raise OSError
return mask.value # преобразование ctypes.c_ulong => python int
def set_affinity(pid=0, mask=1):
mask = ctypes.c_ulong(mask)
c_ulong_size = ctypes.sizeof(ctypes.c_ulong)
if __setaffinity(pid, c_ulong_size, mask) < 0:
raise OSError
return
Apparently, ctypes itself implicitly dealt with the pointer. It is also worth noting that a call with pid = 0 is performed on the current process.
Windows XP +
The documentation for the functions we need indicates:
Minimum supported client - Windows XP Minimum supported server - Windows Server 2003 DLL - Kernel32.dll
Now we know when it will work and which library should be loaded.
We do it by analogy with the Linux version. Take the headlines:
BOOL WINAPI SetProcessAffinityMask(
__in HANDLE hProcess,
__in DWORD_PTR dwProcessAffinityMask
);
BOOL WINAPI GetProcessAffinityMask(
__in HANDLE hProcess,
__out PDWORD_PTR lpProcessAffinityMask,
__out PDWORD_PTR lpSystemAffinityMask
);
As a HANDLE, ctypes.c_uint is quite suitable for us, but you need to be careful with the out parameter types:
DWORD_PTR is the same ctypes.c_uint, and P DWORD_PTR is ctypes.POINTER (ctypes.c_uint).
Total we get:
__setaffinity = ctypes.windll.kernel32.SetProcessAffinityMask
__setaffinity.argtypes = [ctypes.c_uint, ctypes.c_uint]
__getaffinity = ctypes.windll.kernel32.GetProcessAffinityMask
__getaffinity.argtypes = [ctypes.c_uint, ctypes.POINTER(ctypes.c_uint), ctypes.POINTER(ctypes.c_uint)]
And it seems that we’ll do it there and it will work:
def get_affinity(pid=0):
mask_proc = ctypes.c_uint(0)
mask_sys = ctypes.c_uint(0)
if not __getaffinity(pid, mask_proc, mask_sys):
raise ValueError
return mask_proc.value
def set_affinity(pid=0, mask=1):
mask_proc = ctypes.c_uint(mask)
res = __setaffinity(pid, mask_proc)
if not res:
raise OSError
return
But alas . Functions accept not pid, but HANDLE of process. It still needs to be received. To do this, we use the OpenProcess function and the CloseHandle “paired” to it :
PROCESS_SET_INFORMATION = 512
PROCESS_QUERY_INFORMATION = 1024
__close_handle = ctypes.windll.kernel32.CloseHandle
def __open_process(pid, ro=True):
if not pid:
pid = os.getpid()
access = PROCESS_QUERY_INFORMATION
if not ro:
access |= PROCESS_SET_INFORMATION
hProc = ctypes.windll.kernel32.OpenProcess(access, 0, pid)
if not hProc:
raise OSError
return hProc
Without going into details, we just get the HANDLE of the process we need with access to read the parameters, and when ro = False, and to change them. This is written in the documentation for SetProcessAffinityMask and GetProcessAffinityMask:
SetProcessAffinityMask: hProcess [in] A handle to the process whose affinity mask is to be set. This handle must have the PROCESS_SET_INFORMATION access right. GetProcessAffinityMask: hProcess [in] A handle to the process whose affinity mask is desired. Windows Server 2003 and Windows XP: The handle must have the PROCESS_QUERY_INFORMATION access right.
So no Monte Carlo method :)
Rewrite our get_affinity and set_affinity to reflect the changes:
def get_affinity(pid=0):
hProc = __open_process(pid)
mask_proc = ctypes.c_uint(0)
mask_sys = ctypes.c_uint(0)
if not __getaffinity(hProc, mask_proc, mask_sys):
raise ValueError
__close_handle(hProc)
return mask_proc.value
def set_affinity(pid=0, mask=1):
hProc = __open_process(pid, ro=False)
mask_proc = ctypes.c_uint(mask)
res = __setaffinity(hProc, mask_proc)
__close_handle(hProc)
if not res:
raise OSError
return
WindowsXP + for the lazy
To slightly reduce the amount of code for the Win implementation, you can install the pywin32 module . It saves us from having to set constants and deal with libraries and call parameters. Our code above might look something like this:
import win32process, win32con, win32api, win32security
import os
def __open_process(pid, ro=True):
if not pid:
pid = os.getpid()
access = win32con.PROCESS_QUERY_INFORMATION
if not ro:
access |= win32con.PROCESS_SET_INFORMATION
hProc = win32api.OpenProcess(access, 0, pid)
if not hProc:
raise OSError
return hProc
def get_affinity(pid=0):
hProc = __open_process(pid)
mask, mask_sys = win32process.GetProcessAffinityMask(hProc)
win32api.CloseHandle(hProc)
return mask
def set_affinity(pid=0, mask=1):
try:
hProc = __open_process(pid, ro=False)
mask_old, mask_sys_old = win32process.GetProcessAffinityMask(hProc)
res = win32process.SetProcessAffinityMask(hProc, mask)
win32api.CloseHandle(hProc)
if res:
raise OSError
except win32process.error as e:
raise ValueError, e
return mask_old
Briefly, clearly, but this is a third-party module.
And what is the result?
If you put it all together and add one more to our initial tests:
def test3():
cpuinfo.affinity.set_affinity(0,1) # меняем в своем процессе (pid=0) affinity на первое ядро.
test2()
par2 = timeit.timeit( 'test3()', 'from __main__ import test3', number=trying )/trying
print par2
then the results will be as follows:
Linux: test1: 10.41 | 102.89 test2: 13.25 | 135.29 test3: 10.45 | 104.51 Windows: test1: 19.25 | 191.97 test2: 27.41 | 269.78 test3: 19.52 | 196.17
The numbers in the second column - cover those tests, but cnt 10 times Used on lshim.
We got two threads of execution with almost no loss in speed compared to the single-threaded version.
Affinity is set by a bit mask on both OSs. On a 4-core machine, get_affinity returns 15 (1 + 2 + 4 + 8).
An example and all the code for the article posted on github .
I accept any suggestions and complaints.
Also interested in the results on a processor with support for HT and other versions of Linux.
All from the first of April! This code really works :)
