Deploying Django to Elastic Beanstalk with HTTPS redirects and functional health checks

In this technical blog post aimed for fellow AWS developers we will take a look at into securing web applications, more specifically Django applications, using HTTP to HTTPS redirects with Apache on the Amazon Web Services platform. We will also set up functional Elastic Load Balancer health checks for the whole Elastic Beanstalk application we are running.

These instructions are somewhat generic and can be applied to other web stacks as well, such as Java running with Tomcat and Node.js running with nginx.

A foreword about HTTPS and Elastic Beanstalk issues

A common problem with web apps running on Elastic Beanstalk is that they have a Elastic Load Balancer in front of them, but expose both the standard HTTP (80) and HTTPS (443) protocol ports, of which the first one is insecure to use. Some website users default to the first port and send unencrypted traffic to the service, whereas they should be always using HTTPS. The only case where this is not preferable is when users are merely browsing content without sending any data to the web service at all, which is very rare. Even then the cost of HTTPS is marginal. The newer HTTP 2.0 protocol implementations from Google and various web server vendors fix this by commonly not allowing unencrypted HTTP connections at all, but HTTP 1.1 is still commonly in use. The ideal solution would be to disallow all unencrypted HTTP traffic to protect the user, but that would probably scare away potential customers that are not technologically adept about the specifics of encryption and do not check the HTTPS URL if their browser will not automatically redirect them to one.

Another problem is that many Elastic Beanstalk applications do not use health checking at all to allow load balancers or proxies to determine whether or not a server is running a functional web stack or has a broken setup with non-functional components in the mix. Broken setups can be the cause of a deteriorated user experience or security vulnerabilities. A web application should always be able to determine its own health and either not run at all or run in a deterministic downgraded mode with no broken components in use. Either it works as intended, or it does not work at all. Health checking issues are commonplace, and are hard to do with HTTPS enabled stacks, because Elastic Load Balancer needs a bare HTTP connection for determining an application's health. Here we will expose just one URL from our page for such checks, while keeping the rest secured with HTTPS redirects.

Health checking support

The first step is to configure our Django application to check its own health. We start by installing the django-health-check package and configure a health check URL and component checkers for it.

First we add the module to our requirements file:

# requirements.txt
django-health-check

Then install from CLI with pip:

$ pip install -r requirements.txt

Then configure our website settings:

# settings.py
INSTALLED_APPS += (
    'health_check',
    'health_check.db',
    'health_check.cache',
    'health_check.storage',
)

And finally add health checking addresses to our URL configuration:

# urls.py
urlpatterns += patterns(
    '',
    url(r'^health/?', include('health_check.urls')),
)

Go ahead and check that the /health URL works on our development server:

    $ curl -vvv localhost:${DJANGO_DEV_SERVER_PORT:-8000}/health

We should be getting back a bare HTTP response with code

  • HTTP 200 OK, if everything is fine, or;
  • HTTP 500 INTERNAL SERVER ERROR, if something is wrong.

This way, we have a working system for testing out database, caching, Celery, etc. components from out of the box. The django-health-check package is very easy to extend to test other configurations and E2E functionalities as well.

After we have configured our health check URL, it is time to configure the ELB to use it. Let’s start by allowing the load balancer to access our instances. Create an AWS configuration file for Django and put the following into it:

# -*- coding: utf-8 -*-
# AWS configuration file for the project.
# Assumes there is a production.py configuration file
# in the same module / folder to import base settings from.

from django.core.exceptions import ImproperlyConfigure

import requests

from .production import *

def get_ec2_hostname():
    try:
        ipconfig = 'http://169.254.169.254/latest/meta-data/local-ipv4'
        return requests.get(ipconfig, timeout=10).text
    except Exception:
        error = 'You have to be running on AWS to use AWS settings'
        raise ImproperlyConfigured(error)

ALLOWED_HOSTS = [
    get_ec2_hostname(),
    '127.0.0.1',
    'localhost',
    '.amazonaws.com',
    '.elasticbeanstalk.com',
    'myapp.example.com',
]

CSRF_COOKIE_SECURE = True
SESSION_COOKIE_SECURE = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

Above we fetch the instance IP used by the ELB from the internal AWS metadata API and add it our allowed addresses to allow ELB to use it for health checking purposes. Without the local IPv4 address our development server replies HTTP 500 as it thinks the ELB is querying it with a bogus request. Here we are also allowing traffic from the *.amazonaws.com and *.elasticbeanstalk.com domains. You might wish to use stricter settings at your own discretion.

As you see, we won’t be testing storage backends in health checks with the django-health-check package’s built in checkers. As of August 2016 there is ongoing work on the Boto storage interface which is used to access stored files. These cause errors in the default storage module for django-health-check. There are custom modules that can be used to substitute for the health check. You can also write your own health checking module.

Go ahead and update your Elastic Beanstalk deployment. We want to check if our page works as intended with this configuration.

HTTP to HTTPS

We will also want to configure our site to direct its HTTP traffic to HTTPS. We will do this by using custom Apache configurations. Apache is called httpd in EC2 Linux due to its Red Hatted roots, hence the configuration path. We configure Apache by using the following .ebextensions file, which could be named e.g. .ebextensions/00_httpd_django.config:

Go ahead and check if your site forwards traffic from HTTP to HTTPS automatically, and ignores the forwards on /health URL. If it does, great! We need that view to answer in just HTTP 200 OK instead of a HTTP 301 REDIRECT.

This rule sends a HTTP 301 redirection to every non-HTTPS request, which can be the non-referred option if you need direct access inside to the server inside the VPC. HTTPS traffic goes through the ELB. In trivial cases without health checking this can, however, be the easiest option.

Configure the Elastic Load Balancer

The next step is to configure your ELB to check the health by configuring it from the EB interface. Just input the /health URL and enjoy!

If you get any errors, downgrade the health checks back to TCP pings by putting an empty URL on the Health check URL in EB configurations. If the site doesn’t respond with HTTP 200, you won’t see any instances. ELB is taking them out of the server pool because it thinks they are broken.

The true state of your instances is easy to check by using eb ssh command from the awsebcli package to access your servers and checking if localhost:80 responds nicely to traffic requests with e.g.

    $ curl -vvv localhost:80

Some problems you might have here are that:

  • Your server is redirecting you to HTTPS, check the /health URL as well. If you get HTTP 301 REDIRECT for HTTP GET, your ELB thinks your site is broken!
  • Your server is responding nicely to localhost, but ELB thinks its broken. Your ALLOWED_HOSTS setting is probably wrong and your Django server won’t accept the HTTP Host header it is getting. Check that localhost is included, as well as your server IPs.
  • Nothing responds. Your Apache or WSGI setup isn’t valid. Check the logs with eb logs to see what’s wrong.

After everything is configured nicely, we have a Django site that:

  • Is running securely behind HTTPS and forwards traffic over to the secure protocol, keeping users safe.
  • Checks its own health with ELB and won’t receive traffic if it detects any errors in configuration. This way we avoid running broken servers and configurations.
  • Can be scaled with a few button clicks to use a larger database and more instances. Bigger caches and background task runners are available as normal AWS services. Anything can be installed on EC2 instances to run your software. This is a major advantage over e.g. Heroku when you have to start worrying about non-stock components and network configurations with custom services such as ElasticSearch and the like.

Learn more

  • Django is a robust and open-source web framework that runs, among other things, parts of Instagram and Pinterest.
  • Elastic Beanstalk is a managed platform that runs on the same infrastructure Amazon uses for its own web stores and services.

Want to ask about AWS or Django development?

Contact us and let's have a chat!

Aleksi Häkli

Aleksi Häkli

2 kommenttia

HJS says:

Hello,

This tutorial is exactly what I’m looking for. However, I’m having some trouble with it. Do you know if it is up to date with current (September 2017) AWS? I’m attempting this on:

– 64bit Amazon Linux 2017.03 v2.5.1 running Python 3.4
– Django 1.11.2

Things appear to go well until trying to get the HTTP to HTTPS redirect to work. At this point, get errors around Apache not running when trying to redeploy.

ohelal says:

I used the below for Apache 2 and put it in the .conf file I created for my vhost
RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule !/health https://%{HTTP_HOST}%{REQUEST_URI} [L,R]

Also make sure you DON’T have SECURE_SSL_REDIRECT set to true in your django settings.
Also check your custom middleware that may be rejecting certain useragents etc
Hope that helps

Great article by the way, was really helpful!

Join the conversation