Guided Lab: Understanding Visibility Timeout in Amazon SQS
Description
Visibility Timeout in Amazon SQS determines the amount of time a message remains hidden from other consumers after it has been retrieved from the queue. During this period, other consumers cannot process the same message, allowing the current consumer time to complete processing. If the message is not deleted within this timeout, it becomes visible again and can be reprocessed. This mechanism is crucial for preventing multiple consumers from processing the same message simultaneously and for ensuring reliable message processing in case of failures.
In this lab, you will create an SQS queue, set a visibility timeout, and observe how message processing behaves when the timeout is exceeded.
Prerequisites
This lab assumes you have the following:
- Basic knowledge and understanding of Amazon SQS and AWS Lambda
If you find any gaps in your knowledge, consider taking the following lab:
Objectives
By the end of this lab, you will:
- Understand the concept and importance of SQS visibility timeout.
- Learn how to configure the visibility timeout for an SQS queue.
- Understand how messages behave with successful processing within the visibility timeout.
- Observe the behavior when message processing exceeds the visibility timeout and messages are redelivered.
Lab Steps
Create an SQS Queue
1. Navigate to the Amazon SQS service:
2. Create a new queue using the following configurations:
- Click on Create Queue.
- Choose Standard as the queue type and configure the following:
- Queue Name:
MyVisibilityTimeoutQueue
- Queue Name:
- Under Configuration, set the Default Visibility Timeout to
30 seconds
.
- Leave the other settings at their default values and click Create Queue.
Create a Lambda Function to Simulate Message Processing
1. Navigate to AWS Lambda: Search for “Lambda” in the AWS Management Console.
2. Create a new Lambda function using the following configurations:
- Choose Author from scratch.
- Function name:
myLambdaFunction
- Select
Python 3.12
as the runtime. - Execution role:
- Select Use an Existing Role:
PlayCloud-Sanbox
- Select Use an Existing Role:
-
Click Create function
3. Replace the existing code with the following:
import time
def lambda_handler(event, context):
for record in event['Records']:
print("Processing message:", record['body'])
time.sleep(45) # Simulate processing delay (longer than visibility timeout)
print("Finished processing")
This delay (45 seconds
) exceeds the queue’s visibility timeout (30 seconds
), allowing us to test message redelivery.
4. Click Deploy to save the changes.
5. Go to the Configurations tab > General configuration > click Edit and change the Timeout to 30 sec
6. In the Amazon SQS Console, go to MyMainQueue and configure the Lambda function as a trigger:
- Go to Lambda triggers and click Configure Lambda function trigger.
- Choose
myLambdaFunction
as the function and Save. - Wait for the status to change to Enabled.
Send a Message to the Queue
1. Select MyVisibilityTimeoutQueue and click on Send and receive messages.
2. In the Message body field, enter a sample message (e.g., { "order_id": "12345" }
).
3. Click Send message to add it to the queue.
Observe Message Behavior with Visibility Timeout
1. After adding the trigger, the Lambda function will attempt to process the message.
- Since the function’s processing time (
45 seconds
) is longer than the visibility timeout (30 seconds
), the message will reappear in the queue after 30 seconds. - Wait for a few minutes.
- Observe the
MyVisibilityTimeoutQueue
queue in the SQS console.
- Go to the CloudWatch Logs Console to view the logs for the
myLambdaFunction
function. You should see the message being processed multiple times as it becomes visible again due to the visibility timeout.
Modify the Lambda Function to Simulate Successful Processing
1. Return to the AWS Lambda Console and open the lambda function created earlier.
2.Modify the function code with:
import json
def lambda_handler(event, context):
for record in event['Records']:
print("Successfully processed message:", record['body'])
return {
'statusCode': 200,
'body': 'Success'
}
This updated Lambda function processes messages from an SQS queue without the previous delay (sleep
). It iterates over each record in the event, logs the message content, and returns a 200
status with "Success"
to confirm processing. This setup is now optimized for faster processing without any intentional delay.
3. Click Deploy to save changes.
Observe the Message Result
1. After updating the Lambda Function code, check the SQS Queue again. The Message in flight should be Zero ( 0 ) by now.
2. Go to the CloudWatch Logs Console to view the logs for the myLambdaFunction
function again. Open the latest log stream to see the latest log event. You should see successful message and no repeated processing logs.
That’s it. Congratulations! You have successfully tested the Amazon SQS visibility timeout feature by observing both successful message processing within the timeout and redelivery when the timeout expires. Understanding visibility timeout behavior is crucial for reliable message processing, especially for handling scenarios where messages may need to be retried if not processed on time.
Key Takeaways:
- If processing exceeds the visibility timeout, messages reappear for reprocessing.
- Successfully processed messages within the visibility timeout are deleted from the queue.
- Adjusting visibility timeout settings can help manage processing duration to ensure messages are only redelivered when necessary.
As a final step, remember it’s always a best practice to clean up resources after completing a lab. Deleting unused resources will help maintain a clutter-free AWS environment.
Happy learning!!!