POSH: Python Object Sharing

Steffen Viken Valvåg <steffenv@fast.no> - Åge Kvalnes <aage@cs.uit.no> - Kjetil Jacobsen <kjetilja@cs.uit.no>
University of Tromsø, N-9037 Tromsø, Norway

Abstract:

Python uses a single global lock known as the global interpreter lock (or GIL) to serialize execution of byte codes. The GIL becomes a major bottleneck when executing multi-threaded Python applications, in particular on multi-processor architectures. This paper presents POSH, which is an extension module to Python that attempts to address the problems associated with the GIL by enabling placement of Python objects in shared memory. In particular, POSH allows multiple processes to share objects in much the same way that threads do with standard Python objects. We have found that the use of POSH allows some applications to be structured as if they used threads, but without the GIL bottleneck.