2.4 Restrictions on Shared Objects

In general, almost any kind of Python object can be shared between processes using POSH. However, there are a few restrictions that shareable types should obey.

• Shareable types should not override the low-level memory allocation methods tp_alloc and tp_free. These are low-level methods that can only be overridden by extension types implemented in C. Their task is to manage the allocation and deallocation of the memory required for the type's instances. POSH relies on overriding these methods in order to allocate the objects in shared memory.

• Shareable types may not override the attribute lookup methods __ getattribute__,
  __ setattr__ or __ delattr__. POSH implements attributes for shared objects by associating a shared dictionary with each shared object that supports attributes. The dictionary provides storage for the object's attributes and is analogous to the __ dict__ attribute of regular objects. POSH overrides the attribute lookup methods of shareable types with versions that employ the shared dictionary. Shareable types may still provide a custom __ getattr__ method, which is a fall-back method that gets invoked whenever an AttributeError would normally be raised. In addition, shareable types retain the capability to customize their attributes with special attribute descriptors.

• A related restriction, which will probably be lifted in later versions of POSH, is that shareable types cannot have a nonempty __ slots__ attribute. Future versions of POSH can allow this by means of special attribute descriptors, but the feature has not yet been implemented.

• Methods of shareable types should never store the self argument in any way that makes it persist beyond the lifetime of the method call. For instance, it should never be stored in a global variable. The reason for this is that the self argument refers directly to the shared object, and not to a proxy object, which is normally the case when shared objects are accessed. Without the protection of a proxy object, the lifetime of the shared object cannot be guaranteed beyond the duration of the method call. This is because proxy objects play a vital part in the garbage collection algorithm implemented by POSH, in keeping track of the existing references to shared objects. If the self argument were to be stored in a global variable, the garbage collection algorithm would lose track of the shared object and might delete it prematurely.

  If the self argument needs to be stored for later reference, the value stored should be posh.share(self), which wraps the object in a proxy object. Given this small inconvenience, one might wonder why POSH does not automatically wrap the self argument in a proxy object. The answer is simple -- the self argument should always be an instance of the class to which a method belongs. The proxy objects created by POSH are all instances of a common type, and passing a proxy object in place of self would thus violate the semantics of the language, in addition to producing a number of odd side effects.

Most of these restrictions are pretty clear, and do not pose significant limitations in practice. However, the last point is rather subtle, and requires a certain vigilance on the programmer's part to avoid. POSH is able to issue warnings if a type is violating any of the other restrictions, but enforcing the last one remains up to the programmer.