Django and Oscar
Mike Slinn

Django Asset Settings

Published 2021-03-14.
Time to read: 5 minutes.

This page is part of the django collection.

To style a Django web application, assets (also known as static assets) are required, such as images, JavaScript and CSS / SCSS. This article discusses:

  • Settings for establishing where Django looks for assets.
  • Assets provided by Python dependencies.
  • A Django software tool for discovering where assets are served from.

A previous article walks though the process of preparing AWS to provide data storage services to a Django webapp, and another article shows how to switch between a local collection of assets and an asset collection stored on AWS.

Definitions

Media assets
User uploaded files. Not discussed in this article, but the docs referenced do encompass that topic.
Static assets
CSS, JavaScript, images, etc. required for the web application to function properly.

Django Asset Locations

Django assets for a webapp can be stored in many locations:

  • Dependent Python modules. These static files have the lowest precedence.
  • In the static/ directory of each app, which is not created by default. Apps are searched for assets in the order that they are listed in INSTALLED_APPS in settings.
  • In a top-level webapp directory of static assets (there is no default location). Files placed in this directory have the highest precedence.
  • Content delivery networks like Alibaba CDN, AWS CloudFront, CDN77, cdnjs, CloudFlare, Fastly, Google Cloud, IBM CDN, KeyCDN, StackPath, Microsoft Azure CDN, and many more.

Django settings control where assets will be searched for.

The SCRIPT_NAME Environment Variable

This variable is important because it forms part of the directory path to assets.

The WSGI standard can be interpreted to mean that the SCRIPT_NAME environment variable points to a deployed Django webapp. From the WSGI standard:

The initial portion of the request URL’s “path” that corresponds to the application object, so that the application knows its virtual “location”. This may be an empty string, if the application corresponds to the “root” of the server.

Is this true? Why are there two names for one thing?
Setting FORCE_SCRIPT_NAME is how to set the value of SCRIPT_NAME. This ensures that the directory that defines the web application for a Django web application is one level down from the root of the webapp.

Portion of settings/base.py
FORCE_SCRIPT_NAME = "blah/"

Is this true?
Setting FORCE_SCRIPT_NAME to a slash causes a Django web application"s guts to be located in the top-level Django directory.

Portion of settings/base.py
FORCE_SCRIPT_NAME = "/"

Django Asset Settings

The django.contrib.staticfiles app provides support for various directories containing static assets. It does this by supporting settings that define where static assets are looked for, where generated assets are placed, and provides software tools for manipulating assets.

A separate static directory is possible for production, located outside of the Django webapp. Static files are collected and copied or linked to this directory (this is an output directory.) This directory is normally set up so nginx or Apache httpd can serve static files such as CSS, JavaScript and images, or so those files can be synchronized with a directory in a CDN. The STATIC_ROOT setting defines the location of this directory.

Portion of settings/base.py
STATIC_ROOT = '/var/sites/myWebApp/collected_static_assets/'

Within a Django webapp, every app can have its own static/ directory.

  • For development mode, the django.contrib.staticfiles app modifies the manage.py runserver subcommand so static files inside each app’s static/ directory (myProject/myAppName/static/) are served automatically.
  • For production mode, the manage.py collectstatic subcommand collects the contents of the static/ directories and places them in the directory pointed to by STATIC_ROOT.

It is also possible to collect assets from other directories. The STATICFILES_DIRS setting can be defined to hold an array of directory names to search for assets. The following adds a top-level static/ directory in the Django webapp to the asset search path. This directory could be used to hold static files that are common to several apps.

Excerpt from settings/base.py
from pathlib import Path

STATICFILES_DIRS = [
    BASE_DIR / 'static/',
]

Serving Assets

The STATIC_URL setting defines the base URL for serving static assets associated with STATIC_ROOT.

The STATIC_URL setting can be a URL:

Excerpt from settings/base.py
STATIC_URL = 'https://my_bucket.s3.amazonaws.com/'

It can also be a top-level directory path within the Django app:

Excerpt from settings/base.py
STATIC_URL = '/static/'

The manage.py collectstatic Subcommand

Shell
(aw) $ ./manage.py collectstatic -h
usage: manage.py collectstatic [-h] [--noinput] [--no-post-process] [-i PATTERN] [-n] [-c] [-l] [--no-default-ignore]
                               [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback]
                               [--no-color] [--force-color] [--skip-checks]

Collect static files in a single location.

optional arguments:
  -h, --help            show this help message and exit
  --noinput, --no-input
                        Do NOT prompt the user for input of any kind.
  --no-post-process     Do NOT post process collected files.
  -i PATTERN, --ignore PATTERN
                        Ignore files or directories matching this glob-style pattern. Use multiple times to ignore more.
  -n, --dry-run         Do everything except modify the filesystem.
  -c, --clear           Clear the existing files using the storage before trying to copy or link the original file.
  -l, --link            Create a symbolic link to each file instead of copying.
  --no-default-ignore   Don’t ignore the common private glob-style patterns (defaults to "CVS", ".*" and "*~").
  --version             show program’s version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g. "myproject.settings.main". If this isn’t provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g. "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don’t colorize the command output.
  --force-color         Force colorization of the command output.
  --skip-checks         Skip system checks.

Defining Assets

By default, collectstatic copies all files from directories containing static assets.

Storage Provider

Django supports storing and retrieving assets to and from various online storage providers. The STATICFILES_STORAGE setting defines the storage provider. Check out the django-storages GitHub project.

Install the boto3 and django-storages dependencies:

Shell
(aw) $ pip install boto3 django-storages

The most interesting storage provider for me is the AWS S3 storage provider. The previous article walked though the process of preparing AWS to provide data storage services to a Django webapp. If you have not read that article yet, please do so now, then come back here and continue.

AWS S3 Storage Provider Settings

Immediately below are the AWS S3 storage provider settings I found to be of most interest. Note that I do not include AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY because boto3 will use credentials from ~/.aws/credentials if present, just like aws cli:

Portion of settings/base.py
AWS_STORAGE_BUCKET_NAME = 'assets.mywebsite.com'
AWS_LOCATION = 'static'

AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
AWS_QUERYSTRING_AUTH = False

STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'

The documentation has a pointed note about directory syntax.

Django’s STATIC_URL must end in a slash, and AWS_S3_CUSTOM_DOMAIN must not.

Django AWS Classsloader Bug

😁

I believe I found a bug in the Django AWS asset collection implementation. The content-related classloaders are not invoked, so the content is only loaded by the primary classloader. The result is that none of your web content customizations get loaded. I experienced this as a critical path / stop-other-work-and-fix-it-now bug. Here is my workaround.

Referencing Assets

In templates, the static tag builds URLs for a given relative path from a storage provider.

Excerpt from a template
{ % load static %}
<img src="{% static 'my_app/example.jpg' %}" alt="My image">

Django provides many built-in tags and filters for templates.

Collecting Assets

Uploading to AWS

Now that AWS is configured to collect assets, they are automatically uploaded using aws s3 sync when collectstatic is run. In this mode, no assets are copied locally to STATIC_ROOT.

Shell
(aw) $ ./manage.py collectstatic
Using AWS datastore for assets.
(Input) STATICFILES_DIRS=['/var/work/django/frobshop/static']
(Output) STATIC_ROOT=/var/work/django/frobshop/collected_static_assets

You have requested to collect static files at the destination
location as specified in your settings.

This will overwrite existing files!
Are you sure you want to do this? yes

(aw) $ aws s3 ls assets.ancientwarmth.com
PRE admin/
                           PRE debug_toolbar/
                           PRE django_extensions/
                           PRE django_tables2/
                           PRE oscar/
                           PRE treebeard/

Invalidating AWS CloudFront Distribution

After running the collectstatic subcommand, the CloudFront distribution must be invalidated before the changes to the deployed assets become visible. I wrote a bash script called collectstatic that performs the invalidation after calling manage.py collectstatic.

collectstatic 
#!/bin/bash

function waitForInvalidation {
  echo "Waiting for invalidation $2 to complete."
  aws cloudfront wait invalidation-completed \
    --distribution-id "$1" \
    --id "$2"
  echo "Invalidation $2 has completed."
}

function invalidateCloudFrontDistribution {
  set -b

  DIST_ID="$( aws cloudfront list-distributions \
    --query "DistributionList.Items[*].{id:Id,origin:Origins.Items[0].Id}[?origin=='S3-$BUCKET_NAME'].id" \
    --output text
  )"

  JSON="$( aws cloudfront create-invalidation \
    --distribution-id "$DIST_ID" \
    --paths "/*"
  )"

  INVALIDATION_ID="$( jq -r .Invalidation.Id <<< "$JSON" )"
  waitForInvalidation "$DIST_ID" "$INVALIDATION_ID" &
}



unset FORCE_LOCAL_STORAGE
if [ "$1" == --force-local-storage ]; then
  shift
  FORCE_LOCAL_STORAGE=--force-local-storage
fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
else
  BUCKET_NAME=assets.ancientwarmth.com
fi

FORCE_LOCAL_STORAGE_SAVE=$FORCE_LOCAL_STORAGE

# Workaround for https://github.com/jschneier/django-storages/issues/991
# Delete this line when fixed
FORCE_LOCAL_STORAGE=--force-local-storage

./manage.py collectstatic $FORCE_LOCAL_STORAGE \
  --verbosity 0 \
  --noinput \
  --clear

# Workaround for https://github.com/jschneier/django-storages/issues/991
# Delete this command when fixed
aws s3 sync \
  --acl public-read \
  --delete \
  --quiet \
  collected_static_assets/ s3://assets.ancientwarmth.com

if [ -z "$FORCE_LOCAL_STORAGE_SAVE" ]; then invalidateCloudFrontDistribution; fi

Here is an example of using it:

Shell
(aw) $ bin/collectstatic
Using AWS datastore for assets.
(Input) STATICFILES_DIRS=['/var/work/django/frobshop/static']
(Output) STATIC_ROOT=/var/work/django/frobshop/collected_static_assets
{
    "Location": "https://cloudfront.amazonaws.com/2020-05-31/distribution/EIHVJABCDE5K/invalidation/I39XR7TA5LP3NP",
    "Invalidation": {
        "Id": "I39XR7TA5LP3NP",
        "Status": "InProgress",
        "CreateTime": "2021-03-15T17:17:54.349Z",
        "InvalidationBatch": {
            "Paths": {
                "Quantity": 1,
                "Items": [
                    "/*"
                ]
            },
            "CallerReference": "cli-1615828671-931250"
        }
    }
}

Finding Assets

It can be confusing to try to understand where an asset is being served from, or why it is not being served, even when serving from local datastores. The findstatic subcommand of manage.py can help with that problem.

findstatic ignores Django data stores and assumes local assets, just as if the --force-local-storage option is always specified.

findstatic Help

Shell
(aw) $ ./manage.py findstatic -h
usage: manage.py findstatic [-h] [--first] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH]
                            [--traceback] [--no-color] [--force-color] [--skip-checks]
                            staticfile [staticfile ...]

Finds the absolute paths for the given static file(s).

positional arguments:
  staticfile

optional arguments:
  -h, --help            show this help message and exit
  --first               Only return the first match for each static file.
  --version             show program’s version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g. "myproject.settings.main". If this isn’t provided,
                        the DJANGO_SETTINGS_MODULE environment variable will be used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g. "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don’t colorize the command output.
  --force-color         Force colorization of the command output.
  --skip-checks         Skip system checks

Using findstatic

You can discover the absolute path of specific assets, by supplying the load paths. The following displays the absolute path of the asset loaded by admin/js/core.js:

Shell
(aw) $ ./manage.py findstatic admin/js/core.js
Found "admin/js/core.js" here:
    /var/work/django/oscar/lib/python3.8/site-packages/django/contrib/admin/static/admin/js/core.js

You can also obtain the absolute path of an relative asset directory The following displays the absolute path for assets loaded by admin/js/:

Shell
(aw) $ ./manage.py findstatic admin/js
Found "admin/js" here:
    /var/work/django/oscar/lib/python3.8/site-packages/django/contrib/admin/static/admin/js

We can list all the asset files in the absolute directory like this:

Shell
(aw) $ ls `./manage.py findstatic admin/js | tail -n 1`
SelectBox.js      change_form.js   popup_response.js
SelectFilter2.js  collapse.js      prepopulate.js
actions.js        collapse.min.js  prepopulate.min.js
actions.min.js    core.js          prepopulate_init.js
admin             inlines.js       urlify.js
autocomplete.js   inlines.min.js   vendor
calendar.js       jquery.init.js
cancel.js         nav_sidebar.js


* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.