Sending Messages via AWS SQS triggering Lambda using Python

Greetings,

I came back again to show you another project on my 15th week at LUIT. This time I was tasked to work on a project with AWS SQS using Python and Lambda.

What is SQS?

Amazon Simple Queue Service (Amazon SQS) offers a secure, durable, and available hosted queue that lets you integrate and decouple distributed software systems and components. Amazon SQS offers common constructs such as dead-letter queues and cost allocation tags. It provides a generic web services API that you can access using any programming language that the AWS SDK supports.

Source: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html

What is Lambda?

Lambda is a compute service that lets you run code without provisioning or managing servers. Lambda runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, and logging. With Lambda, you can run code for virtually any type of application or backend service. All you need to do is supply your code in one of the languages that Lambda supports.

You organize your code into Lambda functions. Lambda runs your function only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time that you consume — there is no charge when your code is not running.

You can invoke your Lambda functions using the Lambda API, or Lambda can run your functions in response to events from other AWS services. For example, you can use Lambda to:

• Build data-processing triggers for AWS services such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.
• Process streaming data stored in Amazon Kinesis.
• Create your own backend that operates at AWS scale, performance, and security.

Lambda is a highly available service. For more information, see the AWS Lambda Service Level Agreement

Source: https://docs.aws.amazon.com/lambda/latest/dg/welcome.html

Scenario:

1. Create a Standard SQS Queue using Python.
2. Create a Lambda function in the console with a Python 3.7 or higher runtime.
3. Modify the Lambda to send a message to the SQS queue. The message should contain either the current time or a random number, can use the built-in test function for testing.
4. Create an API gateway HTTP API type trigger.
5. Test the trigger to verify the message was sent.

Requirements:

1. Access to AWS Management Console (preferably free tier).
2. Access to an IDE space to execute the code.
3. Git Hub account if required for later access.

For starters login to AWS Management and Launch Cloud9 to go to my IDE and enter the following code and run to create SQS:

To verify login to AWS Management Console and can see the SQS was created successfully as follows:

Next, We need to create an IAM role that will allow Lambda function to be triggered when receiving message from SQS:

Next, created a Lambda function from AWS Management Console as follows:

As shown below the function is created:

When I try to set up Lambda Trigger under SQS notice I got an error :

Couldn’t configure trigger for queue.

Error code: InvalidParameterValueException. Error message: The provided execution role does not have permissions to call ReceiveMessage on SQS.

This is because the Lambda function created doesn’t have permission to receive SQS messages.

In order to grant permission, we need to create a role:

We need to create an In-line Policy.

Create policy:

Then we will replace the code in Lambda with the policy I just created for SQS as follows:

Then I modified Lambda code above with the code below:

code we need to replace the original code by AWS

This will test the code and create a test event name:

We can see Lambda function is now successfully created:

JSON code for the message ID above

Below shows the time when the message was sent:

Next task, I will create an API Gateway that will trigger Lambda function when a user invokes API-URL.

creating API-Gateway

API-URL

As can be see above created API Gateway endpoint, however got error below because I forgot to give permission to Lambda to work with API Gateway.

error before assigning API Gateway a role

Again head back to IAM role to assign AmazoneAPIGateWayAdministrator role. All I can say it all services I have worked so far need permission from IAM (What an Important service) 👌

granting Lambda API-Gateway role

message sent

Sent a message as follows:

message sent

MessageId shown in JSON, which shows the message was sent successfully.

{
  "MD5OfMessageBody": "864800be51f96bc1f958cd96dea36b7d",
  "MessageId": "9e829b2a-8277-4718-9bae-4309a853b2c2",
  "ResponseMetadata": {
    "RequestId": "83a3c0a2-4e94-5bac-a5e9-9b513d90a890",
    "HTTPStatusCode": 200,
    "HTTPHeaders": {
      "x-amzn-requestid": "83a3c0a2-4e94-5bac-a5e9-9b513d90a890",
      "date": "Sun, 20 Nov 2022 00:11:48 GMT",
      "content-type": "text/xml",
      "content-length": "378"
    },
    "RetryAttempts": 0
  }
}

At last , the message sent can be seen below:

Conclusion:

This was real world project which gave me a good understanding Lambda can be automated to do a lot of tasks using Python, it also thought me not to forget to assign a permission before I even start triggering Lambda. Please provide your feedback and suggestions.

Thank you,

Completed!