mark.dog

mark.dog


Currently working at FIFA Index and Klippa. Find me on Twitter, GitHub or LinkedIn – Mark Doggen

October 2019
M T W T F S S
« Jan    
 123456
78910111213
14151617181920
21222324252627
28293031  

Categories


Redis cache decorator

Cache time consuming Python methods

markmark

Some time ago I had to perform some time intensive operations in Python. A certain method would always return the same (given the same arguments), but get queried numerous times while running the script. Caching would make this script a lot faster, so I made this easy Redis decorator that can be slapped onto any Python method and make it’s output cacheable right away.

So, as an example, this might be some method that you’re trying to cache:

def my_cached_function(argument):
    time.sleep(10)  # Some long running operation
    return argument * 100

Just a simple method that returns it’s input value times 100, but you get it, this method can be anything. Now, every call to this method (given the same argument) would cost a tiny bit more than ten seconds. We don’t want that. we only want the very first call to take those ten seconds, and that consequent calls with the same argument return from cache directly. To do this, we add the following decorator:

import redis
import hashlib

r = redis.Redis(unix_socket_path='/etc/redis.sock')

def get_key(func, args):
    key_string = str(func.__name__) + '_' + str(args)
    return hashlib.md5(key_string).hexdigest()

def redis(func):
    def wrapped(*args, **kwargs):
        key = get_key(func, args)
        obj = r.get(key)
        if obj is None:
            obj = func(*args, **kwargs)
            r.set(key, obj)
        return obj
    return wrapped

The redis method is the actual decorator, and it uses to the get_key method to distill a key based on the method name that is being cached and the arguments that it’s being passed. The get_key method builds a md5 hash of these input values that is being used as a key for Redis. This makes sure that any successive calls of the same method, using the same arguments, will refer to the same key within the Redis cache.

Now, to use this as a decorator for our time consuming method, we just use it as a decorator:

@redis
def my_cached_function(argument):
    time.sleep(10)  # Some long running operation
    return argument * 100

And it’s cached!

Note: this works for basic value types (like integers and strings). You might want to serialize any other values like lists or dictionaries before storing them with Redis.