De Mystifying Decorators in Python

What is a decorator ? Types of decorators ?

What is the order of execution of a decorator ?

Can we pass arguments to the decorators ?

Can there be multiple decorators ? What is the order of execution in this case ?

What are various built in decorators available in python?

What is the real time use cases when we need to use decorators ?

efining Decorators:

Python is a pretty special language in that functions are first class objects. What that means is once a function is defined in a scope it can be passed to functions, assigned to variables, even returned from functions. By saying this a decorator is just a function that returns a function. Python decorators add functionality to functions and methods at definition time, they are not used to add functionality at run time. Decorators were introduced in Python 2.4, so be sure your code will be run on >= 2.4.

Lets understand this with a simple example:

def my_decorator(function_to_decorate):
def wrapper():
print('Entering', function_to_decorate.__name__)
function_to_decorate()
print('Exiting', function_to_decorate.__name__)
return wrapper
def my_func():
print("Original function.")
func = my_decorator(my_func)
func()

So in the code above , my_decorator has taken my_func as an argument. and returns another inner function which actually calls my_func. So when we call func() it actually calls the inner wrapper function which in turn calls my_func.

Instead of all the above , Let’s make it shorter:

def my_decorator(function_to_decorate):
def wrapper():
print('Entering', function_to_decorate.__name__)
function_to_decorate()
print('Exiting', function_to_decorate.__name__)
return wrapper
@my_decorator
def my_func():
print("Original function.")
my_func()

If you observe closely wrapper() is able to use and remembers function_to_decorate parameter which was actually passed as an argument to the my_decorator. Here wrapper() is a closure. Visual representation is depicted as below:

A closure is an anonymous function that refers to its parameters or other variables outside its scope. So decorators uses closures.

ifferent Types of Decorators

Similar to the above function decorators we can have class decorators also.We can define a decorator as a class in order to do that, we have to use a __call__ method of classes. When a user needs to create an object that acts as a function then function decorator needs to return an object that acts like a function, so __call__ can be useful.

More info on __call__ as follows:

__init__ is called when you are creating an instance of any class and initializing the instance variable also.

Example:

class User:

def __init__(self,first_n,last_n,age):
self.first_n = first_n
self.last_n = last_n
self.age = age

user1 = User("Jhone","Wrick","40")

And __call__ is called when you call the object like any other function.

Example:

class USER:
def __call__(self,arg):
"todo here"
print(f"I am in __call__ with arg : {arg} ")

user1=USER()
user1("One") #calling the object user1 and that's gonna call __call__ dunder functions

Example1:

class MyDecorator:
def __init__(self, function):
self.function = function

def __call__(self):
self.function()
@MyDecorator
def function():
print("Sabya")
function()

By far we must have understood that in order to call an object as a normal function we need to __call__. Similarly function() in the above needs an object that acts as a function, we need __call__ to implemented.

Example2: (Class Decorator with return statement)

class SquareDecorator:
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
result = self.function(*args, **kwargs)
return result
@SquareDecorator
def get_square(n):
print("given number is:", n)
return n * n
print("Square of number is:", get_square(195))

Example3: (Checking error parameter using class decorator)

class ErrorCheck:
def __init__(self, function):
self.function = function
def __call__(self, *params):
if any([isinstance(i, str) for i in params]):
raise TypeError("parameter cannot be a string !!")
else:
return self.function(*params)
@ErrorCheck
def add_numbers(*numbers):
return sum(numbers)
#returns 6
print(add_numbers(1, 2, 3))
#raises Error.
print(add_numbers(1, '2', 3))

Order of execution of decorator:

Python decorators add functionality to functions and methods at definition time, they are not used to add functionality at run time.

What does this mean :

class SquareDecorator:
def __init__(self, function):
print("Square decorator init called")
self.function = function
def __call__(self, *args, **kwargs):
result = self.function(*args, **kwargs)
return result
@SquareDecorator
def get_square(n):
print("given number is:", n)
return n * n
#####print("Square of number is:", get_square(195))

In the code above even if we have commented the calling of get_square . As soon as we define the SquareDecorator and @SquareDecorator , __init__ would be called . And it is true for function decorators also.

Can we pass arguments to the decorators ?

Yes we can pass arguments to the decorators . Below is the code for the same:

def decorator_maker_with_arguments(decorator_arg1, decorator_arg2):

print("I make decorators! And I accept arguments {} {}".format(decorator_arg1, decorator_arg2))

def my_decorator(func):
# The ability to pass arguments here is a gift from closures.
# If you are not comfortable with closures, you can assume it’s ok,
# or read: http://stackoverflow.com/questions/13857/can-you-explain-closures-as-they-relate-to-python
print("I am the decorator. Somehow you passed me arguments {} {}".format(decorator_arg1, decorator_arg2))

# Don’t confuse decorator arguments and function arguments!
def wrapped(function_arg1, function_arg2):
print ('I am the wrapper around the decorated function.\n'
'I can access all the variables\n'
'\t- from the decorator: {0} {1}\n'
'\t- from the function call: {2} {3}\n'
'Then I can pass them to the decorated function'
.format(decorator_arg1, decorator_arg2,
function_arg1, function_arg2))
return func(function_arg1, function_arg2)

return wrapped

return my_decorator
@decorator_maker_with_arguments('SABYA', 'SACHI')
def decorated_function_with_arguments(function_arg1, function_arg2):
print ('I am the decorated function and only knows about my arguments: {0}'
' {1}'.format(function_arg1, function_arg2))

decorated_function_with_arguments('WORKING', 'IN PYTHON')
#outputs:
I make decorators! And I accept arguments SABYA SACHI
I am the decorator. Somehow you passed me arguments SABYA SACHII am the wrapper around the decorated function.I can access all the variables- from the decorator: SABYA SACHI- from the function call: WORKING IN PYTHONThen I can pass them to the decorated functionI am the decorated function and only knows about my arguments: WORKING IN PYTHON

Can there be multiple decorators wrapped a single function? What is the order of execution in this case ?

def decorator_a(func):
print 'Get in decorator_a'
def inner_a(*args, **kwargs):
print 'Get in inner_a'
return func(*args, **kwargs)
return inner_a

def decorator_b(func):
print 'Get in decorator_b'
def inner_b(*args, **kwargs):
print 'Get in inner_b'
return func(*args, **kwargs)
return inner_b

@decorator_b
@decorator_a
def f(x):
print 'Get in f'
return x * 2

f(1)

the final execution result is:

Get in decorator_a
Get in decorator_b
Get in inner_b
Get in inner_a
Get in f

The decorator function is executed immediately after the decorator function is defined. And the execution order is decorated from bottom to top. When decorator_a is called, f is decorated as inner_a, and when decorator_b is called, f is decorated as inner_b.

So in the code shown above, when f(1) is executed last, f has become inner_b, and the func of return in inner_b is actually inner_a, and the func of return in inner_a is the final f.

So the last call order is

inner_b — ->inner_a — ->f

What are various built in decorators available in python?

@staticmethod , @classmethod, @property , @functools.wraps

What is the real time use cases when we need to use decorators ?

Usecase1: Analytics, logging, and instrumentation

Especially with large applications, we often need to specifically measure what’s going on, and record metrics that quantify different activities.

from myapp.log import logger

def log_order_event(func):
def wrapper(*args, **kwargs):
logger.info("Ordering: %s", func.__name__)
order = func(*args, **kwargs)
logger.debug("Order result: %s", order.result)
return order
return wrapper

@log_order_event
def order_pizza(*toppings):
# let's get some pizza!

Usecase2: Validation and runtime checks

Let’s say we have a application that users can log into and interact with a nice gui (graphical user interface). The user’s interactions with the gui trigger events that cause Python functions to get executed. Let’s assume that there are lots of users using this application and that they have a bunch of different permission levels. The execution of different functions requires different permission types. For example consider the following functions:

#assume these functions exist
def current_user_id():
"""
this function returns the current logged in user id, if the user is not authenticated then return None
"""

def get_permissions(iUserId):
"""
returns a list of permission strings for the given user. For example ['logged_in','administrator','premium_member']
"""
#we need to implment permission checking on these functions

def delete_user(iUserId):
"""
delete the user with the given Id. This function is only accessable to users with administrator permissions
"""

def new_game():
"""
any logged in user can start a new game
"""

def premium_checkpoint():
"""
save the game progress, only accessable to premium members
"""

One way to implement these permissions would be to make multiple decorators, for example:

def requires_admin(fn):
def ret_fn(*args,**kwargs):
lPermissions = get_permissions(current_user_id())
if 'administrator' in lPermissions:
return fn(*args,**kwargs)
else:
raise Exception("Not allowed")
return ret_fn
def requires_logged_in(fn):
def ret_fn(*args,**kwargs):
lPermissions = get_permissions(current_user_id())
if 'logged_in' in lPermissions:
return fn(*args,**kwargs)
else:
raise Exception("Not allowed")
return ret_fn

def requires_premium_member(fn):
def ret_fn(*args,**kwargs):
lPermissions = get_permissions(current_user_id())
if 'premium_member' in lPermissions:
return fn(*args,**kwargs)
else:
raise Exception("Not allowed")
return ret_fn

@requires_admin
def delete_user(iUserId):
"""
delete the user with the given Id. This function is only accessable to users with administrator permissions
"""
@requires_logged_in
def new_game():
"""
any logged in user can start a new game
"""

@requires_premium_member
def premium_checkpoint():
"""
save the game progress, only accessable to premium members
"""

But that’s pretty horrible. It requires a lot of copy-paste, and each decorator requires a different name, and if any change is made to how permissions are checked then every decorator has to be updated. Wouldn’t it be great to have one decorator that does the job of all three?

To do this we need a function that returns a decorator:

def requires_permission(sPermission):                            
def decorator(fn):
def decorated(*args,**kwargs):
lPermissions = get_permissions(current_user_id())
if sPermission in lPermissions:
return fn(*args,**kwargs)
raise Exception("permission denied")
return decorated
return decorator


def get_permissions(iUserId): #this is here so that the decorator doesn't throw NameErrors
return ['logged_in',]
def current_user_id(): #ditto on the NameErrors
return 1
#and now we can decorate stuff... @requires_permission('administrator')
def delete_user(iUserId):
"""
delete the user with the given Id. This function is only accessible to users with administrator permissions
"""
@requires_permission('logged_in')
def new_game():
"""
any logged in user can start a new game
"""

@requires_permission('premium_member')
def premium_checkpoint():
"""
save the game progress, only accessable to premium members
"""

Challenges in decorator :

def mydeco(func):
def wrapper(*args, **kwargs):
return f'{func(*args, **kwargs)}!!!'
return wrapper

Let’s now decorate two different functions with “mydeco”:

@mydeco
def add(a, b):
'''Add two objects together, the long way'''
return a + b
@mydeco
def mysum(*args):
'''Sum any numbers together, the long way'''
total = 0
for one_item in args:
total += one_item
return total

But there are a few issues with what we did. For example, what if I ask each function for its name:

>>> add.__name__
'wrapper'
>>> mysum.__name__
'wrapper'

The __name__ attribute, which gives us the name of a function when we define it, now reflects the returned internal function, “wrapper”, in our decorator. Now, this might be true, but it’s not helpful.

We can solve this problem, at least partially, by assigning to the __name__ and __doc__ attributes in our decorator:

def mydeco(func):
def wrapper(*args, **kwargs):
return f'{func(*args, **kwargs)}!!!'
wrapper.__name__ = func.__name__
wrapper.__doc__ = func.__doc__
return wrapper
>>> help(add)
Help on function add in module __main__:

add(*args, **kwargs)
Add two objects together, the long way
>>> help(mysum)
Help on function mysum in module __main__:

mysum(*args, **kwargs)
Sum any numbers together, the long way

The good news is that we’ve now fixed the naming and the docstring problem. But the function signature is still that super-generic one, looking for both *args and **kwargs.

The solution, as people reminded me after my talk, is to use functools.wraps. It’s designed to solve precisely these problems.

from functools import wraps

def mydeco(func):
@wraps(func)
def wrapper(*args, *kwargs):
return f'{func(*args, **kwargs)}!!!'
return wrapper

Link to Second chapter on Demystifying Decorators