How to Configure a Django Application with Amazon S3 Buckets for Static and Media Files(File Storage)

Published in

AWS Tip

8 min readAug 16, 2023

Amazon’s Simple Storage System (S3) provides a simple, cost-effective way to store static files. This tutorial shows how to configure Django to load and serve up static and user uploaded media files, public and private, via an Amazon S3 bucket.

In many server-side programming languages or web frameworks, handling static files (like images, CSS, and JS) and user-uploaded content is a common task. The traditional approach of serving these files directly from the file system can become inefficient, especially as the application scales. This is due to the added load on the server processes that are already handling user requests.

To address this, leveraging cloud file services, such as Amazon S3, can offer several advantages. With cloud storage, files can be stored centrally and accessed by multiple applications, promoting reusability. Additionally, using cloud-based storage like S3 can significantly enhance performance and reduce loading times, as specialized content delivery mechanisms can be employed.

By the end of this tutorial, you will be able to:

Serve static files from S3 buckets
Serve media files from S3 buckets
Upload and retrieve files from S3 buckets

The main dependencies include:

Django any version
Python v3…

Step 1: Create an AWS Account

Before beginning, you will need an AWS account. If you’re new to AWS, Amazon provides a free tier with 5GB of S3 storage.

To create an S3 bucket, navigate to the S3 page and click “Create bucket”:

Give the bucket a unique, DNS-compliant name and select a region:You can leave most of the configuration as default.

Scroll down to the Block Public Access settings for this bucket section, uncheck the Block all public access setting, and acknowledge the warning that shows up.

Once done, click the Create bucket button. It will redirect you to a page that shows a list of the S3 buckets you have created.

Create an IAM User on AWS

Although you could use the AWS root user, it’s best for security to create an IAM (Identity and Access Management) user that only has access to S3 or to a specific S3 bucket. What’s more, by setting up a group, it makes it much easier to assign (and remove) access to the bucket.

For security purposes, you should create an IAM user for your Django project to interact with your S3 bucket. Follow these steps to create an IAM user on AWS:

In the search bar, type IAM and select the first option. A new page will appear.

2. On the left-hand side of the IAM page, select Users, then proceed to click the Add users button. It will open up another page to fill in some information.

3. Start by entering a name for the IAM user and clicking the Next button at the bottom:

On the next page, you must select the permission levels for the IAM user. Follow these steps:

First, select the Attach policies directly option from the Permissions options section.
Next, define a permission policy for your IAM user. This will determine what the IAM user can and cannot do. Since you want your Django app to download and upload files, you should give it full access to the S3 bucket.
In the Permissions policies section, you should search for S3FullAccess and select the option. Once done, click the Next button.

4. Next, review the policies for the IAM user and click the Create user button to create your IAM user.

Create an Access Key for Your IAM User

In AWS, an access key refers to credentials you can use to authenticate and securely access AWS resources programmatically. Your Django project must provide these credentials to access your S3 bucket.

The following steps will help you generate an access key for your project.

After creating your IAM user, you’ll get an alert message that prompts you to view the user. Alternatively, you can view the user by clicking on the User name.

2. Next, select the Security credentials tab, scroll down to find the Access keys section, and select Create access key.

3. You’ll need to select a use case for your access, so AWS can recommend an alternative option where appropriate. It does not affect your access key. Feel free to select an option such as Third-party service or Local code and acknowledge the warning that pops up. Once done, click the Next button.

4. On the next page, enter a description tag for your access key and click the Create access key button.

5. After creating your access key, you can either copy your credentials or download them as a CSV file. Either way, make sure to keep this data safe and secure.

Configure Your Django Project for S3 Bucket

To use your S3 bucket with a Django project, install these packages:

django-storages: This package will help you define a storage backend for your files.
boto3: This package is the AWS Software Development Kit (SDK) to help your Python project interact with AWS.

You can install these packages into your Python virtual environment with Python’s Pip package manager by typing this command in your terminal:

pip install django-storages boto3

Once you have successfully installed these packages, open your settings.py file and add boto3 to the installed apps.

The last thing to do is configure your Django project to use the AWS S3 bucket. Here’s the general configuration to use:

AWS_ACCESS_KEY_ID = 'AWS_ACCESS_KEY_ID '
AWS_SECRET_ACCESS_KEY = 'AWS_SECRET_ACCESS_KEY'
AWS_STORAGE_BUCKET_NAME = 'AWS_STORAGE_BUCKET_NAME'
AWS_S3_SIGNATURE_NAME = 's3v4',
AWS_S3_REGION_NAME = 'AWS_S3_REGION_NAME'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
AWS_S3_VERITY = True
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

Paste the above configuration in your settings.py file and replace the values accordingly. Replace your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with the access key and secret access key you copied or downloaded earlier. You should also change the AWS_STORAGE_BUCKET_NAME and AWS_S3_REGION_NAME to the names of your S3 bucket and region.

You can get the region name by navigating to your S3 bucket and copying the last values from the AWS region column.

Step 6: Test Your AWS Configuration

With the above steps complete, you should be ready to test your application by uploading files. The following code samples will upload files directly from the admin panel, but you’re free to upload yours from another place.

For context, you can have a model that looks like this:

class Post(models.Model):
    title = models.CharField(max_length=225, blank=False, null=False)
    content = models.TextField('Post Body', blank=False, null=False)
    author = models.CharField(max_length=225, blank=False, null=False)
    date_published = models.DateTimeField(auto_now=True)
    image = models.ImageField(upload_to='posts')

def __str__(self):
        return self.title

Ensure you perform the necessary operations such as migrations, adding it to the admin panel, creating a view, and other things necessary for your project. Ensure you practice Django’s MVT principle.

Once you’re done, navigate to your admin panel or whatever form you have created for file upload, and upload an image file.

Navigate to your main site and confirm the image is there. If it is, right-click on the image and select the Open image in new tab option. In the new tab containing the image, you’ll notice that the address bar references the S3 bucket you created earlier:

Another way to confirm your configuration is working is by navigating to your bucket on the AWS console. You will find your image there:

Collect Static Files to Your S3 Bucket

So far, you have been able to upload media files to your S3 bucket; now, you need to upload your static files.

To do that, add these configurations to your settings.py file:

STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_LOCATION = 'static'

After that, open up your Command Line Interface (CLI) and run this command:

python manage.py collectstatic --noinput

To confirm that everything works, open your S3 bucket in the AWS console. You will see a folder called static.

Use AWS S3 Bucket for Much More

The possibilities of AWS S3 buckets are enormous! You should familiarize yourself with it and learn how to use S3 for other purposes, like hosting a static web application.

Knowing how to properly use S3 buckets will save you a lot of time and help you build a better product or solution. But you should also be aware that S3 might not be suitable for every case, so consider your project requirements before using it.

Conclusion

his tutorial walked you through how to create a bucket on Amazon S3, configure an IAM user and group, and set up Django to upload and serve static files and media uploads to and from S3.

By using S3, you:

Increase the amount of space you have available for static and media files
Decrease the stress on your own server since it no longer has to serve up the files
Can limit access to specific files

References

That’s it, I guess.

Cheers, Happy Coding!!!

🙏🙏🙏

Since you’ve made it this far, sharing this article on your favorite social media network would be highly appreciated. For feedback, please ping me on Twitter.