[Solved] – Python – Usage of __slots__?

Usage of __slots__?

Asked on October 13, 2018 in Python.
Add Comment


  • 4 Answer(s)

    The special attribute slots permits you to expressly state that instance attributes you expect your object instances to own, with the expected results:

    faster attribute access.
    space savings in memory.
    The house savings is from

    Storing worth references in slots rather than alcoholic.
    Denying alcoholic and weakref creation if parent categories deny them and you declare slots.
    Quick Caveats
    Small caveat, you ought to solely declare a selected slot just the once in AN inheritance tree. For example:

    class Base:
        __slots__ = 'foo', 'bar'
    class Right(Base):
        __slots__ = 'baz',
    class Wrong(Base):
        __slots__ = 'foo', 'bar', 'baz' # redundant foo and bar
    

    Python does not object once you get this wrong (it in all probability should), issues won’t otherwise manifest, however your objects can take up more room than they otherwise ought to.

    >>> from sys import getsizeof
    getsizeof(Right()), getsizeof(Wrong())
    (64, 80)
    

    The biggest caveat is for multiple inheritance – multiple “parent categories with nonempty slots” can not be combined.

    To accommodate this restriction, follow best practices: work out near one or all parents’ abstraction that their concrete category severally and your new concrete category put together can inherit from – giving the abstraction(s) empty slots (just like abstract base categories within the customary library).

    See section on multiple inheritance below for AN example.

    Requirements:
    To have attributes named in slots to really be hold on in slots rather than a alcoholic, a category should inherit from object.

    To prevent the creation of a alcoholic, {you should|you want to|you need to} inherit from object and every one categories within the inheritance must declare slots and none of them will have a ‘dict’ entry.

    There area unit a great deal of details if you would like to stay reading.

    Why use slots: quicker attribute access.
    The creator of Python, Guido van Rossum, states that he truly created slots for quicker attribute access.

    It is trivial to demonstrate measurably important quicker access:
    It is trivial to demonstrate measurably significant faster access:

    import timeit
    class Foo(object): __slots__ = 'foo',
    class Bar(object): pass
    slotted = Foo()
    not_slotted = Bar()
    def get_set_delete_fn(obj):
        def get_set_delete():
            obj.foo = 'foo'
            obj.foo
            del obj.foo
        return get_set_delete

    and

    >>> min(timeit.repeat(get_set_delete_fn(slotted)))
    0.2846834529991611
    min(timeit.repeat(get_set_delete_fn(not_slotted)))
    0.3664822799983085
    

    The slotted access is almost 30% faster in Python 3.5 on Ubuntu.

    >>> 0.3664822799983085 / 0.2846834529991611
    1.2873325658284342
    

    Why use slots: Memory Savings
    Another purpose of slots is to cut back the house in memory that every object instance takes up.
    My own contribution to the documentation clearly states the explanations behind this:
    The house saved over exploitation inveterate will be vital.
    SQLAlchemy attributes plenty of memory savings to slots.
    To verify this, exploitation the Eunectes murinus distribution of Python a pair of.7 on Ubuntu UNIX operating system, with guppy.hpy (aka heapy) and sys.getsizeof, the scale of a category instance while not slots declared, and zip else, is sixty four bytes. That doesn’t embrace the inveterate. thanks Python for lazy analysis once more, the inveterate is outwardly not referred to as into existence till it’s documented, however categories while not information area unit typically useless. once referred to as into existence, the inveterate attribute may be a minimum of 280 bytes to boot.
    In distinction, a category instance with slots declared to be () (no data) is merely sixteen bytes, and fifty six total bytes with one item in slots, sixty four with 2.
    For sixty four bit Python, I illustrate the memory consumption in bytes in Python a pair of.7 and 3.6, for slotsand inveterate (no slots defined) for every purpose wherever the dict grows in three.6 (except for zero, 1, and a couple of attributes):

    Python 2.7 Python 3.6
    attrs __slots__ __dict__* __slots__ __dict__* | *(no slots defined)
    none 16 56 + 272† 16 56 + 112† | †if __dict__ referenced
    one 48 56 + 272 48 56 + 112
    two 56 56 + 272 56 56 + 112
    six 88 56 + 1040 88 56 + 152
    11 128 56 + 1040 128 56 + 240
    22 216 56 + 3344 216 56 + 408
    43 384 56 + 3344 384 56 + 752

    So, in spite of smaller dicts in Python 3, we see how nicely slots scale for instances to save us memory, and that is a major reason you would want to use slots.
    Just for completeness of my notes, note that there is a one-time cost per slot in the class’s namespace of 64 bytes in Python 2, and 72 bytes in Python 3, because slots use data descriptors like properties, called “members”.

    >>> Foo.foo
    <member 'foo' of 'Foo' objects>
    type(Foo.foo)
    <class 'member_descriptor'>
    getsizeof(Foo.foo)
    72

    Demonstration of slots:
    To deny the creation of a dict, you must subclass object:

    class Base(object):
        __slots__ = ()

    now:

    >>> b = Base()
    b.a = 'a'
    Traceback (most recent call last):
        File "<pyshell#38>", line 1, in <module>
            b.a = 'a'
    AttributeError: 'Base' object has no attribute 'a'

    Or subclass another class that defines slots

    class Child(Base):
        __slots__ = ('a',)

    and now:

    c = Child()
    c.a = 'a

    but:

    >>> c.b = 'b'
    Traceback (most recent call last):
        File "<pyshell#42>", line 1, in <module>
            c.b = 'b'
    AttributeError: 'Child' object has no attribute 'b'

    To allow dict creation while subclassing slotted objects, just add ‘dict‘ to the slots (note that slots are ordered, and you shouldn’t repeat slots that are already in parent classes):

    class SlottedWithDict(Child):
        __slots__ = ('__dict__', 'b')
    swd = SlottedWithDict()
    swd.a = 'a'
    swd.b = 'b'
    swd.c = 'c'

     

    and

    >>> swd.__dict__
    {'c': 'c'}

    Or you don’t even need to declare slots in your subclass, and you will still use slots from the parents, but not restrict the creation of a dict:

    class NoSlots(Child): pass
    ns = NoSlots()
    ns.a = 'a'
    ns.b = 'b'

    And:

    >>> ns.__dict__
    {'b': 'b'}

    However, slots may cause problems for multiple inheritance:

    class BaseA(object):
        __slots__ = ('a',)
    class BaseB(object):
        __slots__ = ('b',)

    Because creating a child class from parents with both non-empty slots fails:

    >>> class Child(BaseA, BaseB): __slots__ = ()
    Traceback (most recent call last):
        File "<pyshell#68>", line 1, in <module>
            class Child(BaseA, BaseB): __slots__ = ()
    TypeError: Error when calling the metaclass bases
        multiple bases have instance lay-out conflict

    If you run into this problem, You could just remove slots from the parents, or if you have control of the parents, give them empty slots, or refactor to abstractions:

    from abc import ABC
    class AbstractA(ABC):
        __slots__ = ()
    class BaseA(AbstractA):
        __slots__ = ('a',)
    class AbstractB(ABC):
        __slots__ = ()
    class BaseB(AbstractB):
        __slots__ = ('b',)
    class Child(AbstractA, AbstractB):
        __slots__ = ('a', 'b')
    c = Child() # no problem!

    Add ‘dict‘ to slots to get dynamic assignment:

    class Foo(object):
        __slots__ = 'bar', 'baz', '__dict__'

    and now:

    >>> foo = Foo()
    foo.boink = 'boink'

    So with ‘dict‘ in slots we tend to lose a number of the scale edges with the upper side of getting dynamic assignment and still having slots for the names we tend to do expect.
    When you inherit from Associate in Nursing object that won’t slotted, you get identical form of linguistics after you use slots – names that area unit in slots purpose to slotted values, whereas the other values area unit place within the instance’s dependent.
    Avoiding slots as a result of you would like to be able to add attributes on the fly is truly not an honest reason – simply add “dict” to your slots if this can be needed.
    You can equally add weakref to slots expressly if you would like that feature.Set to empty tuple when subclassing a namedtuple:
    The namedtuple builtin make immutable instances that are very lightweight (essentially, the size of tuples) but to get the benefits, you need to do it yourself if you subclass them:

    from collections import namedtuple
    class MyNT(namedtuple('MyNT', 'bar baz')):
        """MyNT is an immutable and lightweight object"""
        __slots__ = ()

    usage:

    >>> nt = MyNT('bar', 'baz')
    nt.bar
    'bar'
    nt.baz
    'baz'

    And trying to assign an unexpected attribute raises an AttributeError because we have prevented the creation of dict:

    >>> nt.quux = 'quux'
    Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
    AttributeError: 'MyNT' object has no attribute 'quux'

    You can allow dict creation by leaving off slots = (), but you can’t use non-empty slots with subtypes of tuple.
    Biggest Caveat: Multiple inheritance
    Even when non-empty slots are the same for multiple parents, they cannot be used together:

    class Foo(object):
        __slots__ = 'foo', 'bar'
    class Bar(object):
        __slots__ = 'foo', 'bar' # alas, would work if empty, i.e. ()
    class Baz(Foo, Bar): pass
    Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
    TypeError: Error when calling the metaclass bases
        multiple bases have instance lay-out conflict

    Using an empty slots in the parent seems to provide the most flexibility, allowing the child to choose to prevent or allow (by adding ‘dict‘ to get dynamic assignment, see section above) the creation of a dict:

    class Foo(object): __slots__ = ()
    class Bar(object): __slots__ = ()
    class Baz(Foo, Bar): __slots__ = ('foo', 'bar')
    b = Baz()
    b.foo, b.bar = 'foo', 'bar'

    You don’t have to have slots – so if you add them, and remove them later, it shouldn’t cause any problems.
    Going out on a limb here: If you’re composing mixins or using abstract base classes, which aren’t intended to be instantiated, an empty slots in those parents seems to be the best way to go in terms of flexibility for subclassers.
    To demonstrate, first, let’s create a class with code we’d like to use under multiple inheritance

    class AbstractBase:
        __slots__ = ()
        def __init__(self, a, b):
            self.a = a
           self.b = b
        def __repr__(self):
            return f'{type(self).__name__}({repr(self.a)}, {repr(self.b)})'

    We could use the above directly by inheriting and declaring the expected slots:

    class Foo(AbstractBase):
        __slots__ = 'a', 'b'
    

    But we don’t care about that, that’s trivial single inheritance, we need another class we might also inherit from, maybe with a noisy attribute:

    class AbstractBaseC:
        __slots__ = ()
        @property
        def c(self):
            print('getting c!')
            return self._c
        @c.setter
        def c(self, arg):
            print('setting c!')
            self._c = arg
    

    Now if both bases had nonempty slots, we couldn’t do the below. (In fact, if we wanted, we could have given AbstractBase nonempty slots a and b, and left them out of the below declaration – leaving them in would be wrong):

    class Concretion(AbstractBase, AbstractBaseC):
        __slots__ = 'a b _c'.split()
    

    And now we have functionality from both via multiple inheritance, and can still deny dict and weakref instantiation:

    >>> c = Concretion('a', 'b')
    c.c = c
    setting c!
    c.c
    getting c!
    Concretion('a', 'b')
    c.d = 'd'
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    AttributeError: 'Concretion' object has no attribute 'd'
    

    Other cases to avoid slots:
    • Avoid them once you need to perform class assignment with another category that does not have them (and you cannot add them) unless the slot layouts area unit identical. (I am terribly inquisitive about learning UN agency is doing this and why.)
    • Avoid them if you would like to taxonomic group variable length builtins like long, tuple, or str, and you would like to feature attributes to them.
    • Avoid them if you enforce providing default values via category attributes as an example variables.
    You may be ready to tease out additional caveats from the remainder of the slots documentation (the three.7 dev docs area unit the foremost current), that I actually have created important recent contributions to.
    Critiques of different answers
    The current prime answers cite noncurrent data and area unit quite hand-wavy and miss the mark in some necessary ways in which.
    Do not “only use slots once instantiating innumerable objects”
    I quote:
    “You would need to use slots if you’re getting to instantiate tons (hundreds, thousands) of objects of a similar category.”
    Abstract Base categories, for instance, from the collections module, don’t seem to be instantiated, however slots area unit declared for them.
    Why?
    If a user desires to deny dependent or weakref creation, those things should not be obtainable within the parent categories.
    slots contributes to reusability once making interfaces or mixins.
    It is true that a lot of Python users are not writing for reusability, however once you area unit, having the choice to deny unnecessary area usage is effective.
    slots does not break pickling
    When pickling a slotted object, you’ll notice it complains with a deceptive TypeError:

    >>> pickle.loads(pickle.dumps(f))
    TypeError: a category that defines __slots__ while not shaping __getstate__ can not be preserved
    

    This is truly incorrect. This message comes from the oldest protocol, that is that the default. you’ll choose the most recent protocol with the -1 argument. In Python two.7 this is able to be two (which was introduced in two.3), and in 3.6 it is 4.

    >>> pickle.loads(pickle.dumps(f, -1))
    <__main__.Foo object at 0x1129C770>

    in Python 2.7:

    >>> pickle.loads(pickle.dumps(f, 2))
    <__main__.Foo object at 0x1129C770>
    

    in Python 3.6

    >>> pickle.loads(pickle.dumps(f, 4))
    <__main__.Foo object at 0x1129C770>
    

    So I would keep this in mind, as it is a solved problem.
    Critique of the (until Oct 2, 2016) accepted answer
    The first paragraph is 0.5 short rationalization, 0.5 prognostic. Here’s the sole half that really answers the question
    The proper use of slots is to avoid wasting house in objects. rather than having a dynamic dict that permits adding attributes to things at anytime, there’s a static structure that doesn’t enable additions when creation. this protects the overhead of 1 dict for each object that uses slots
    The last half is illusion, and off the mark:
    While this is often generally a helpful improvement, it might be fully supererogatory if the Python interpreter was dynamic enough so it might solely need the dict once there really were additions to the thing.
    Python really will one thing almost like this, solely making the hooked once it’s accessed, however making numerous objects with no knowledge is fairly ridiculous.
    The second paragraph oversimplifies and misses actual reasons to avoid slots. The below isn’t a true reason to avoid slots (for actual reasons, see the remainder of my answer on top of.):
    They change the behavior of the objects that have slots in a very manner that may be abused by management freaks and static typewriting weenies.
    It then goes on to debate different ways in which of accomplishing that perverse goal with Python, not discussing something to try and do with slots.
    The third paragraph is a lot of illusion. along it’s principally off-the-mark content that the respondent did not even author and contributes to ammunition for critics of the location.
    Memory usage evidence
    Create some normal objects and slotted objects:

    >>> class Foo(object): pass
    class Bar(object): __slots__ = ()
    

    Instantiate a million of them:

    >>> foos = [Foo() for f in xrange(1000000)]
    bars = [Bar() for b in xrange(1000000)]
    

    Inspect with guppy.hpy().heap():

    >>> guppy.hpy().heap()
    Partition of a set of 2028259 objects. Total size = 99763360 bytes.
    Index Count % Size % Cumulative % Kind (class / dict of class)
    0 1000000 49 64000000 64 64000000 64 __main__.Foo
    1 169 0 16281480 16 80281480 80 list
    2 1000000 49 16000000 16 96281480 97 __main__.Bar
    3 12284 1 987472 1 97268952 97 str
    ...
    Admission the steady objects and their __dict__ and examine again:
    for f in foos:
    ... f.__dict__
    guppy.hpy().heap()
    Partition of a set of 3028258 objects. Total size = 379763480 bytes.
    Index Count % Size % Cumulative % Kind (class / dict of class)
    0 1000000 33 280000000 74 280000000 74 dict of __main__.Foo
    1 1000000 33 64000000 17 344000000 91 __main__.Foo
    2 169 0 16281480 4 360281480 95 list
    3 1000000 33 16000000 4 376281480 99 __main__.Bar
    4 12284 0 987472 0 377268952 99 str
    

     

    Answered on October 13, 2018.
    Add Comment

    The proper use of slots is to save lots of house in objects. rather than having a dynamic dict that permits adding attributes to things at anytime, there’s a static structure that doesn’t permit additions when creation. [This use of slots eliminates the overhead of 1 dict for each object.] whereas this can be typically a helpful improvement, it’d be utterly redundant if the Python interpreter was dynamic enough so it’d solely need the dict once there truly were additions to the thing.

    Unfortunately there’s a facet result to slots. they alter the behavior of the objects that have slots in an exceedingly means which will be abused by management freaks and static writing weenies. this can be dangerous, as a result of the management freaks ought to be abusing the metaclasses and also the static writing weenies ought to be abusing decorators, since in Python, there ought to be just one obvious means of doing one thing.

    Making CPython sensible enough to handle saving house while not slots could be a major endeavor, that is maybe why it’s not on the list of changes for P3k (yet).

    Answered on October 13, 2018.
    Add Comment

    You would need to use slots if you’re getting to instantiate plenty (hundreds, thousands) of objects of identical category. slots solely exists as a memory optimisation tool.

    It’s extremely discouraged to use slots for limiting attribute creation, and generally you wish to avoid it as a result of it breaks pickle, together with another musing options of python.

    Answered on October 13, 2018.
    Add Comment

    Another somewhat obscure use of __slots__ is to add attributes to an object proxy from the ProxyTypes package, formerly part of the PEAK project. Its ObjectWrapper allows you to proxy another object, but intercept all interactions with the proxied object. It is not very commonly used (and no Python 3 support), but we have used it to implement a thread-safe blocking wrapper around an async implementation based on tornado that bounces all access to the proxied object through the ioloop, using thread-safe concurrent.Future objects to synchronise and return results.

    By default any attribute access to the proxy object will give you the result from the proxied object. If you need to add an attribute on the proxy object, __slots__ can be used.

    EXAMPLE
    
    from peak.util.proxies import ObjectWrapper
    
    class Original(object):
        def __init__(self):
            self.name = 'The Original'
    
    class ProxyOriginal(ObjectWrapper):
    
        __slots__ = ['proxy_name']
    
        def __init__(self, subject, proxy_name):
            # proxy_info attributed added directly to the
            # Original instance, not the ProxyOriginal instance
            self.proxy_info = 'You are proxied by {}'.format(proxy_name)
    
            # proxy_name added to ProxyOriginal instance, since it is
            # defined in __slots__
            self.proxy_name = proxy_name
    
            super(ProxyOriginal, self).__init__(subject)
    
    if __name__ == "__main__":
        original = Original()
        proxy = ProxyOriginal(original, 'Proxy Overlord')
    # Both statements print "The Original"
        print "original.name: ", original.name
        print "proxy.name: ", proxy.name
    
        # Both statements below print 
        # "You are proxied by Proxy Overlord", since the ProxyOriginal
        # __init__ sets it to the original object 
        print "original.proxy_info: ", original.proxy_info
        print "proxy.proxy_info: ", proxy.proxy_info
    
        # prints "Proxy Overlord"
        print "proxy.proxy_name: ", proxy.proxy_name
        # Raises AttributeError since proxy_name is only set on 
        # the proxy object
        print "original.proxy_name: ", proxy.proxy_name
    Answered on January 14, 2019.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.