Nov. 1, 2014, midnight

Serving large and small files (or anything in between) with Django

Since the beginning of times (1981) a common use of web applications has been serving files, django is capable of that, but is usually not considered the best alternative. But why? The honest answer is: because there are servers that will do this by a fraction of the resources that django takes, but this doesn't mean that you can't do it at all (as long as you know the limitations that you will face).

To be fair, small applications can handle files through django views without much complication, but as your app usage grows, problems will arise. Let's explore three ways of serving a file in a django application.

Load file in memory and serve with django

This first approach is probably the most common, but not much economic, since you load the entire file into memory.

The drawbacks

Load file chunks and serve with django

The drawbacks

Both examples work in small scale, but imagine that you have a lot of concurrent users and many big files to handle, you simply can't afford that much memory and cores.

Let apache (or another webserver) deal with files

If you let a webserver handle the files for you, with their optimised code designed to serve data of granular size, your hardware will use much less memory and CPU cycles. A common approach (with apache) is to use the x-send-file module.

It handles the file transference directly and can be triggered by a django view as following:

You will need to install and enable the x-send-module, in ubuntu systems (tested on 14.04) just type:

apt-get install apache2-mod-xsendfile

a2enmod xsendfile

you also have to add the XSendFile and XSendFilePath directives in your virtual host configuration:

<VirtualHost *:80>
    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
    WSGIScriptAlias / /srv/

    XSendFile On
    XSendFilePath /tmp

If everything goes right, each file requested to your View that match a file in the XSendFilePath will be handled by apache, and you will still be able to programmatically apply any restriction to define who can access the given file.

comments powered by Disqus