Friday, November 5, 2010

Python instance methods - How are they different from ordinary functions?

Python OO model works exceptionally well. Its introspection capabilities, and flexibility provided by the descriptors implementation allow for a lot of interesting thing - one of the nicest being what is available through the "property" built-in call.

However, the descriptor model implies in some behaviors that can raise eyebrows, even for experienced users. Like this bit here:
>>> class A(object):
...   def b(self): pass
...
>>> A.b is A.b
False
>>> a = A()
>>> a.b is a.b
False



Eeeeek!  :-) And now what? Well - it happens that while Object methods are not just functions that receive the object instance as its first parameter - that is, methods are objects from a different class than functions - they are not recorded in the class or in the object instance itself. Rather: both bound and unbound methods for new-style classes in Python are created at attribute retrieval-time: The very "a.b" expression above is, internally, a call to a.__getattribute__("b") - and this call uses the descritor mechanism to wrap the function stored in the class __dict__ attribute in an instance method.

So, methods are objects that contain a reference to the original function, but prepends the "self" variable to the function call when they are called for a bound instance:

   
>>> class A(object):
...   def b(self): pass
>>> a = A()
>>> a.b.im_func
<function b at 0x7f6613a6cd70>
>>> a.b.im_func is A.b.im_func is A.__dict__["b"]
True


(extra bonus: I just learnd about the possibility of concatenating the "is" operator here as well :-) )

This way of doing things works fine, in almost all circunstances. However, the only kinds of objects that are turned into "properties which are method factories" at class creation time are functions. If you want to have other kind of callable objects (like any instance of a class with a "call") into a  proper method, you have to create the descriptor for that yourself. It is actually quite simple -you just have to "implement the descriptor protocol" - which is another way to say, you just have to add a __get__ method :-) . Let's suppose I want a method for a class that is itself an object which keeps track of how many times it was called:
   
class CounterFunction(object):
    """Designed to work as function decorator - will not work for methods """
    def __init__(self, function):
        self.func = function
        self.counter = 0
    def __call__(self, *args, **kw):
        self.counter += 1
        return self.func(*args, **kw)

@CounterFunction
def b(): pass

b()
b()
print b.counter
#-> 2


And if we try to use that as a decorator for a method,
the method will no longer "work":

class C(object):
    @CounterFunction
    def d(self): pass

c = C()
c.d()
#Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
#  File "<stdin>", line 8, in __call__
#TypeError: d() takes exactly 1 argument (0 given)


However a call like "c.d(c) " would succeed  - the c.d object just does not get its instance prepended to the argument list.

If we do, instead:
from types import MethodType

class CounterFunction(object):
    """Designed to work as function or method decorator """
    def __init__(self, function):
        self.func = function
        self.counter = 0
    def __call__(self, *args, **kw):
        self.counter += 1
        return self.func(*args, **kw)
    def __get__(self, instance, owner):
        return MethodType(self, instance, owner)

class C(object):
    @CounterFunction
    def d(self): pass

c = C()
c.d()
print c.d.counter
#-> 1

This "__get__" method, as above, does exactly what the authomatically created "getter" for methods does when we use an ordnary method. This behavior is implemented in the metaclass for Python classes, that is, the "type" method. Whena  class object is instantiaded (the moment it is defined) the __new__ method inside type checks which objects in its body dictionary are functions, and replaces them by the descritors with a __get__ method like the one above.

It should be noted that it is an official advise that you can increase performance in certain code patterns by caching an object method in a local variable rather than retrieveing the method each time you want to call it - that is:

for i in xrange(5000):
    a.do_something()


is actually faster as:

b = a.do_something
for i in xrange(5000):
    b()


The exact behavior of methods and descriptors is described in detail here in the official documentation, here, --look at "user methods"  objects:
http://docs.python.org/reference/datamodel.html

2 comments:

  1. Decorator parece ser bem mais tranqüilo que as annotations do Java.

    ReplyDelete
  2. Thanks for the post, it solved a problem I was having and I've posted it as a solution to a StackOverflow question, too.

    http://stackoverflow.com/questions/9523370/adding-attributes-to-instance-methods-in-python/

    ReplyDelete