Django, Flask, and Redis Tutorial: Web Application Session Management Between Python Frameworks

Django Versus Flask: When Django is the Wrong Choice

I love and use Django in lots of my personal and client projects, mostly for more classical web applications and those involving relational databases. However, Django is no silver bullet.

By design, Django is very tightly coupled with its ORM, Template Engine System, and Settings object. Plus, it’s not a new project: it carries a lot of baggage to remain backwards compatible.

Some Python developers see this as a major problem. They say that Django isn’t flexible enough and avoid it if possible and, instead, use a Python microframework like Flask.

I don’t share that opinion. Django is great when used in the appropriate place and time, even if it doesn’t fit into every project spec. As the mantra goes: “Use the right tool for the job”.

(Even when it is not the right place and time, sometimes programming with Django can have unique benefits.)

In some cases, it can indeed be nice to use a more lightweight framework (like Flask). Often, these microframeworks start to shine when you realize how easy they are to hack on.

Microframeworks to the Rescue

In a few of my client projects, we’ve discussed giving up on Django and moving to a microframework, typically when the clients want to do some interesting stuff (in one case, for example, embedding ZeroMQ in the application object) and the project goals seem more difficult to achieve with Django.

More generally, I find Flask useful for:

• Simple REST API backends
• Applications that don’t require database access
• NoSQL-based web apps
• Web apps with very specific requirements, like custom URL configurations

At the same time, our app required user registration and other common tasks that Django solved years ago. Given its light weight, Flask doesn’t come with the same toolkit.

The question emerged: is Django an all-or-nothing deal? Should we drop it completely from the project, or can we learn to combine it with the flexibility of other microframeworks or traditional frameworks? Can we pick and choose the pieces we want to use and eschew others?

Can we have the best of both worlds? I say yes, especially when it comes to session management.

(Not to mention, there are a lot of projects out there for Django freelancers.)

Now the Python Tutorial: Sharing Django Sessions

The goal of this post is to delegate the tasks of user authentication and registration to Django, yet use Redis to share user sessions with other frameworks. I can think of a few scenarios in which something like this would be useful:

• You need to develop a REST API separately from your Django app but want to share session data.
• You have a specific component that may need to be replaced later on or scaled out for some reason and still need session data.

For this tutorial, I’ll use Redis to share sessions between two frameworks (in this case, Django and Flask). In the current setup, I’ll use SQLite to store user information, but you can have your back-end tied to a NoSQL database (or a SQL-based alternative) if need be.

Understanding Sessions

To share sessions between Django and Flask, we need to know a bit about how Django stores its session information. The Django docs are pretty good, but I’ll provide some background for completeness.

Session Management Varieties

Generally, you can choose to manage your Python app’s session data in one of two ways:

• Cookie-based sessions: In this scenario, the session data is not stored in a data store on the back-end. Instead, it’s serialized, signed (with a SECRET_KEY), and sent to the client. When the client sends that data back, its integrity is checked for tampering and it is deserialized again on the server.

• Storage-based sessions: In this scenario, the session data itself is not sent to the client. Instead, only a small portion is sent (a key) to indicate the identity of the current user, stored on the session store.

In our example, we’re more interested in the latter scenario: we want our session data to be stored on the back-end and then checked in Flask. The same thing could be done in the former, but as the Django documentation mentions, there are some concerns about the security of the first method.

The General Workflow

The general workflow of session handling and management will be similar to this diagram:

Let’s walk through session sharing in a little more detail:

1. When a new request comes in, the first step is to send it through the registered middleware in the Django stack. We’re interested here in the SessionMiddleware class which, as you might expect, is related to session management and handling:

   class SessionMiddleware(object):
   
       def process_request(self, request):
           engine = import_module(settings.SESSION_ENGINE)
           session_key = request.COOKIES.get(settings.SESSION_COOKIE_NAME, None)
           request.session = engine.SessionStore(session_key)
   

   In this snippet, Django grabs the registered SessionEngine (we’ll get to that soon), extracts the SESSION_COOKIE_NAME from request (sessionid, by default) and creates a new instance of the selected SessionEngine to handle session storage.

• Later on (after the user view is processed, but still in the middleware stack), the session engine calls its save method to save any changes to the data store. (During view handling, the user may have changed a few things within the session, e.g., by adding a new value to session object with request.session.) Then, the SESSION_COOKIE_NAME is sent to the client. Here’s the simplified version:

  def process_response(self, request, response):
      ....
  
      if response.status_code != 500:
          request.session.save()
          response.set_cookie(settings.SESSION_COOKIE_NAME,
                  request.session.session_key, max_age=max_age,
                  expires=expires, domain=settings.SESSION_COOKIE_DOMAIN,
                  path=settings.SESSION_COOKIE_PATH,
                  secure=settings.SESSION_COOKIE_SECURE or None,
                  httponly=settings.SESSION_COOKIE_HTTPONLY or None)
  
      return response
  

We’re particularly interested in the SessionEngine class, which we’ll replace with something to store and load data to and from a Redis back-end.

Fortunately, there are a few projects that already handle this for us. Here’s an example from redis_sessions_fork. Pay close attention to the save and load methods, which are written so as to (respectively) store and load the session into and from Redis:

class SessionStore(SessionBase):
    """
    Redis session back-end for Django
    """
    def __init__(self, session_key=None):
        super(SessionStore, self).__init__(session_key)

    def _get_or_create_session_key(self):
        if self._session_key is None:
            self._session_key = self._get_new_session_key()
        return self._session_key

    def load(self):
        session_data = backend.get(self.session_key)
        if not session_data is None:
            return self.decode(session_data)
        else:
            self.create()
            return {}

    def exists(self, session_key):
        return backend.exists(session_key)

    def create(self):
        while True:
            self._session_key = self._get_new_session_key()
            try:
                self.save(must_create=True)
            except CreateError:
                continue
            self.modified = True
            self._session_cache = {}
            return

    def save(self, must_create=False):
        session_key = self._get_or_create_session_key()
        expire_in = self.get_expiry_age()
        session_data = self.encode(self._get_session(no_load=must_create))
        backend.save(session_key, expire_in, session_data, must_create)

    def delete(self, session_key=None):
        if session_key is None:
            if self.session_key is None:
                return
            session_key = self.session_key
        backend.delete(session_key)

It’s important to understand how this class is operating as we’ll need to implement something similar on Flask to load session data. Let’s take a closer look with a REPL example:

>>> from django.conf import settings
>>> from django.utils.importlib import import_module

>>> engine = import_module(settings.SESSION_ENGINE)
>>> engine.SessionStore()
<redis_sessions_fork.session.SessionStore object at 0x3761cd0>

>>> store["count"] = 1
>>> store.save()
>>> store.load()
{u'count': 1}

The session store’s interface is pretty easy to understand, but there’s a lot going on under the hood. We should dig a little deeper so that we can implement something similar on Flask.

Note: You might ask, “Why not just copy the SessionEngine into Flask?” Easier said than done. As we discussed in the beginning, Django is tightly coupled with its Settings object, so you can’t just import some Django module and use it without any additional work.

Django Session (De-)Serialization

As I said, Django does a lot of work to mask the complexity of its session storage. Let’s check the Redis key that’s stored in the above snippets:

>>> store.session_key
u"ery3j462ezmmgebbpwjajlxjxmvt5adu"

Now, lets query that key on the redis-cli:

redis 127.0.0.1:6379> get "django_sessions:ery3j462ezmmgebbpwjajlxjxmvt5adu"
"ZmUxOTY0ZTFkMmNmODA2OWQ5ZjE4MjNhZmQxNDM0MDBiNmQzNzM2Zjp7ImNvdW50IjoxfQ=="

What we see here is a very long, Base64-encoded string. To understand its purpose, we need to look at Django’s SessionBase class to see how it’s handled:

class SessionBase(object):
    """
    Base class for all Session classes.
    """

    def encode(self, session_dict):
        "Returns the given session dictionary serialized and encoded as a string."
        serialized = self.serializer().dumps(session_dict)
        hash = self._hash(serialized)
        return base64.b64encode(hash.encode() + b":" + serialized).decode('ascii')

    def decode(self, session_data):
        encoded_data = base64.b64decode(force_bytes(session_data))
        try:
            hash, serialized = encoded_data.split(b':', 1)
            expected_hash = self._hash(serialized)
            if not constant_time_compare(hash.decode(), expected_hash):
                raise SuspiciousSession("Session data corrupted")
            else:
                return self.serializer().loads(serialized)
        except Exception as e:
            # ValueError, SuspiciousOperation, unpickling exceptions
            if isinstance(e, SuspiciousOperation):
                logger = logging.getLogger('django.security.%s' %
                        e.__class__.__name__)
                logger.warning(force_text(e))
            return {}

The encode method first serializes the data with the current registered serializer. In other words, it converts the session into a string, which it can later convert back into a session (look at the SESSION_SERIALIZER documentation for more). Then, it hashes the serialized data and uses this hash later on as a signature to check the integrity of the session data. Finally, it returns that data pair to the user as a Base64-encoded string.

By the way: before version 1.6, Django defaulted to using pickle for serialization of session data. Due to security concerns, the default serialization method is now django.contrib.sessions.serializers.JSONSerializer.

Encoding an Example Session

Let’s see the session management process in action. Here, our session dictionary will simply be a count and some integer, but you can imagine how this would generalize to more complicated user sessions.

>>> store.encode({'count': 1})
u'ZmUxOTY0ZTFkMmNmODA2OWQ5ZjE4MjNhZmQxNDM0MDBiNmQzNzM2Zjp7ImNvdW50IjoxfQ=='

>>> base64.b64decode(encoded)
'fe1964e1d2cf8069d9f1823afd143400b6d3736f:{"count":1}'

The result of the store method (u’ZmUxOTY…==’) is an encoded string containing the serialized user session and its hash. When we decode it, we indeed get back both the hash (‘fe1964e…’) and the session ({"count":1}).

Note that the decode method checks to ensure that the hash is correct for that session, guaranteeing integrity of the data when we go to use it in Flask. In our case, we’re not too worried about our session being tampered with on the client side because:

• We aren’t using cookie-based sessions, i.e., we’re not sending all user data to the client.

• On Flask, we’ll need a read-only SessionStore which will tell us if given key exists or not and return the stored data.

Extending to Flask

Next, let’s create a simplified version of the Redis session engine (database) to work with Flask. We’ll use the same SessionStore (defined above) as a base class, but we’ll need to remove some of its functionality, e.g., checking for bad signatures or modifying sessions. We’re more interested in a read-only SessionStore that will load the session data saved from Django. Let’s see how it comes together:

class SessionStore(object):

    # The default serializer, for now
    def __init__(self, conn, session_key, secret, serializer=None):

        self._conn = conn
        self.session_key = session_key
        self._secret = secret
        self.serializer = serializer or JSONSerializer

    def load(self):
        session_data = self._conn.get(self.session_key)

        if not session_data is None:
            return self._decode(session_data)
        else:
            return {}

    def exists(self, session_key):
        return self._conn.exists(session_key)


    def _decode(self, session_data):
        """
        Decodes the Django session
        :param session_data:
        :return: decoded data
        """
        encoded_data = base64.b64decode(force_bytes(session_data))
        try:
            # Could produce ValueError if there is no ':'
            hash, serialized = encoded_data.split(b':', 1)
            # In the Django version of that they check for corrupted data
            # I don't find it useful, so I'm removing it
            return self.serializer().loads(serialized)
        except Exception as e:
            # ValueError, SuspiciousOperation, unpickling exceptions. If any of
            # these happen, return an empty dictionary (i.e., empty session).
            return {}

We only need the load method because it’s a read-only implementation of the storage. That means you can’t logout directly from Flask; instead, you might want to redirect this task to Django. Remember, the goal here is to manage sessions between these two Python frameworks to give you more flexibility.

Flask Sessions

The Flask microframework supports cookie-based sessions, which means all of the session data is sent to the client, Base64-encoded and cryptographically signed. But actually, we’re not very interested in Flask’s session support.

What we need is to get the session ID created by Django and check it against the Redis back-end so that we can be sure the request belongs to a pre-signed user. In summary, the ideal process would be (this syncs up with the diagram above):

• We grab the Django session ID from the user’s cookie.
• If the session ID is found in Redis, we return the session matching that ID.
• If not, we redirect them to a login page.

It’ll be handy to have a decorator to check for that information and set the current user_id into the g variable in Flask:

from functools import wraps
from flask import g, request, redirect, url_for

def login_required(f):
    @wraps(f)
    def decorated_function(*args, **kwargs):
        djsession_id = request.cookies.get("sessionid")
        if djsession_id is None:
            return redirect("/")

        key = get_session_prefixed(djsession_id)
        session_store = SessionStore(redis_conn, key)
        auth = session_store.load()

        if not auth:
            return redirect("/")

        g.user_id = str(auth.get("_auth_user_id"))

        return f(*args, **kwargs)
    return decorated_function

In the example above, we’re still using the SessionStore we defined previously to fetch the Django data from Redis. If the session has an _auth_user_id, we return the content from the view function; otherwise, the user is redirected to a login page, just like we wanted.

Gluing Things Together

In order to share cookies, I find it convenient to start Django and Flask via a WSGI server and glue them together. In this example, I’ve used CherryPy:

from app import app
from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

d = wsgiserver.WSGIPathInfoDispatcher({
    "/":application,
    "/backend":app
})
server = wsgiserver.CherryPyWSGIServer(("127.0.0.1", 8080), d)

With that, Django will serve on “/” and Flask will serve on “/backend” endpoints.

In Conclusion

Rather than examining Django versus Flask or encouraging you only to learn the Flask microframework, I’ve welded together Django and Flask, getting them to share the same session data for authentication by delegating the task to Django. As Django ships with plenty of modules to solve user registration, login, and logout (just to name a few), combining these two frameworks will save you valuable time while providing you with the opportunity to hack on a manageable microframework like Flask.