Event-Driven — Automatically Create Amazon Fargate Tasks When Uploading Objects to an S3 Bucket

Automation in performing actions has become a very common and basic requirement when you are using cloud services.

There are some use cases where you want to perform certain actions on some objects like images or documents when the user uploads them. In this case, the actions must be performed automatically. You cannot check each time whether the user has uploaded the object and start some process manually.

You can consider some cases where users upload their profile pictures to your website and images are compressed and stored somewhere.

This article focuses on a similar use case where uploading some object to s3 will create a task on Amazon ECS/Fargate automatically.

Here we will not focus on the task performed by the Amazon Farget task. Our main focus is to run the task(s) when some objects are uploaded to the S3 bucket.

we will use the following services:

• S3 bucket
• S3 Event notification
• Amazon Fargate
• Amazon EventBridge

Prerequisites:

• Create an Amazon Fargate cluster
• Create a task definition using which the task will be created
• Create an S3 bucket

Let’s get started.

Once you create an S3 bucket, you have to enable the bucket to send the event notification to the Amazon EventBridge.

Click on the bucket and open the Properties tab.

At the bottom, you will find Event notifications. Here, beside the Amazon EventBridge, click on Edit and update it to On.

This will allow S3 to send event notifications to Amazon EventBridge.

Once you set up the Amazon Fargate cluster and the task definition is ready, Create a rule in Amazon EventBridge. Click on Create rule.

Mention the details in the Define rule detail. As we want to create a task based on the object uploaded in the S3 bucket, select Rule with an event pattern.

Under the Build event pattern, select the required options and mention the detail.

Select the Event source as AWS events or EventBridge partner events.

Under the Creation method, Select Use pattern form.

Under the Event pattern, select Event source as AWS services. Select AWS service as Simple Storage Service (S3). Select Event type as Amazon S3 Event Notification. Select the Specific events as Object Created, and finally mention the bucket name under Specific bucket(s) by name.

If you want to trigger EventBridge based on uploading an object to a specific folder in the bucket, Edit the Event pattern by clicking on the Edit pattern and add the following lines and replace the folder-name with yours.

{
  "source": ["aws.s3"],
  "detail-type": ["Object Created"],
  "detail": {
    "bucket": {
      "name": ["myb-280689"]
    },
    "object":{
      "key":[{
        "prefix":"folder-name"
      }]
    }
  }
}

Click on next.

Under the Select targets(s), mention the required detail.

Under Target1, select Target type as AWS Service, select Target as ECS task, select the cluster you already created, select the Task definition, and select Latest as we will use the latest version of the Task definition.

Under the compute options, select the Launch type as FARGATE, and under the Configure network configuration, mention the subnet ids of the subnets in which you want to deploy the task.

Keep the remaining information as default, review, and create the rule.

Now upload an object to the S3 bucket of the folder you have mentioned in the event pattern. You can see the task getting created in the ECS cluster.

If this post was helpful, please do follow and click the clap 👏 button below to show your support.

Thanks for reading. Happy learning!

Get an email whenever Aniruddh Fataniya publishes.

In Plain English

Thank you for being a part of our community! Before you go:

• Be sure to clap and follow the writer! 👏
• You can find even more content at PlainEnglish.io 🚀
• Sign up for our free weekly newsletter. 🗞️
• Follow us on Twitter(X), LinkedIn, YouTube, and Discord.