Currently working at Klippa. Find me on Twitter, GitHub or LinkedIn – Mark Doggen

November 2020


Redis cache decorator

Cache time consuming Python methods


Some time ago I had to perform some time intensive operations in Python. A certain method would always return the same (given the same arguments), but get queried numerous times while running the script. Caching would make this script a lot faster, so I made this easy Redis decorator that can be slapped onto any Python method and make it’s output cacheable right away.

So, as an example, this might be some method that you’re trying to cache:

def my_cached_function(argument):
    time.sleep(10)  # Some long running operation
    return argument * 100

Just a simple method that returns it’s input value times 100, but you get it, this method can be anything. Now, every call to this method (given the same argument) would cost a tiny bit more than ten seconds. We don’t want that. we only want the very first call to take those ten seconds, and that consequent calls with the same argument return from cache directly. To do this, we add the following decorator:

import redis
import hashlib

r = redis.Redis(unix_socket_path='/etc/redis.sock')

def get_key(func, args):
    key_string = str(func.__name__) + '_' + str(args)
    return hashlib.md5(key_string).hexdigest()

def redis(func):
    def wrapped(*args, **kwargs):
        key = get_key(func, args)
        obj = r.get(key)
        if obj is None:
            obj = func(*args, **kwargs)
            r.set(key, obj)
        return obj
    return wrapped

The redis method is the actual decorator, and it uses to the get_key method to distill a key based on the method name that is being cached and the arguments that it’s being passed. The get_key method builds a md5 hash of these input values that is being used as a key for Redis. This makes sure that any successive calls of the same method, using the same arguments, will refer to the same key within the Redis cache.

Now, to use this as a decorator for our time consuming method, we just use it as a decorator:

def my_cached_function(argument):
    time.sleep(10)  # Some long running operation
    return argument * 100

And it’s cached!

Note: this works for basic value types (like integers and strings). You might want to serialize any other values like lists or dictionaries before storing them with Redis.