Guided Lab: Configuring Dead Letter Queues (DLQ) in Amazon SQS
Description
A Dead Letter Queue (DLQ) in Amazon Simple Queue Service (SQS) is a secondary queue that stores messages that could not be successfully processed or delivered. DLQs are helpful for tracking and troubleshooting message failures, as they capture messages that are repeatedly processed unsuccessfully. By configuring a DLQ for your primary SQS queue, you can review failed messages separately without impacting the main processing flow.
In this lab, you will create an SQS queue with a DLQ, configure message handling rules, and observe how unprocessed messages are transferred to the DLQ.
Prerequisites
This lab assumes you have the following:
- Basic knowledge and understanding of Amazon SQS, and AWS Lambda
If you find any gaps in your knowledge, consider taking the following lab:
Objectives
By the end of this lab, you will:
- Understand the purpose of Dead Letter Queues in Amazon SQS.
- Learn how to create an SQS queue with a DLQ.
- Configure a DLQ for tracking unprocessed messages and observe its behavior.
Lab Steps
Create the Main SQS Queue
1. Navigate to the Amazon SQS service:
2. Create a new queue using the following configurations:
- Click on Create Queue.
- Choose Standard as the queue type and configure the following:
- Name:
MyMainQueue
- Name:
- Leave the default settings for the rest and click Create Queue.
Create the Dead Letter Queue (DLQ)
1. In the SQS Console, click Create Queue again with the following configuration this time:
- Choose Standard as the queue type and configure:
- Name:
MyDLQ
- Name:
- Leave the default settings and click Create Queue.
Configure the Main Queue to Use the DLQ
1. Go to MyMainQueue and click on Edit.
2. Scroll down to the Dead-letter queue section and Enable it.
- Select MyDLQ as the Dead Letter Queue.
- Set Maximum Receives to
3
(meaning a message will be sent to the DLQ after three unsuccessful processing attempts).
3. Click Save Changes.
Simulate Processing Failures
1. Navigate to AWS Lambda: Search for “Lambda” in the AWS Management Console.
2. Create a new Lambda function using the following configurations:
- Choose Author from scratch.
- Function name:
myLambdaFunction
- Select
Python 3.12
as the runtime. - Execution role:
- Select Use an Existing Role:
PlayCloud-Sanbox
- Select Use an Existing Role:
-
Click Create function
3. Replace the existing code with the following:
import json
def lambda_handler(event, context):
# Simulate failure by always returning an error
print("Simulating message processing failure")
raise Exception("Processing failed")
This Lambda function simulates a failure when processing SQS messages. The lambda_handler
function logs “Simulating message processing failure” to indicate the intentional failure, then raises an exception with “Processing failed.” This stops the function from succeeding, which causes messages to retry and eventually be sent to a Dead Letter Queue (DLQ) after multiple failed attempts. This setup is useful for testing DLQ functionality.
4. Click Deploy to save the changes.
5. In the Amazon SQS Console, go to MyMainQueue and configure the Lambda function as a trigger:
- Go to Lambda triggers and click Configure Lambda function trigger.
- Choose
myLambdaFunction
as the function and Save.
- Wait for the status to change to Enabled.
Send Messages to the Main Queue
1. Select MyMainQueue and click Send and receive messages.
2. In the Message body field, enter a sample message (e.g., { "order_id": "111111111" }
) and click Send message.
3. Send multiple messages to simulate a workload for testing. In this lab, send at least 3-7 messages.
Observe Messages in the DLQ
1. After a few attempts, messages that could not be processed by the Lambda function should automatically move to MyDLQ due to the Maximum Receives setting.
- Return to the Amazon SQS Queue Dashboard and click the refresh icon.
2. Go to MyDLQ > Send and receive message > click Poll for Messages. Check the messages to confirm that failed messages have been transferred. For example:
Congratulations! You have successfully configured a Dead Letter Queue (DLQ) in Amazon SQS to capture messages that fail processing in the main queue. This setup helps isolate problematic messages, making it easier to troubleshoot issues without interrupting the main queue.
Key Takeaways:
- Dead Letter Queues allow for separate handling of failed messages, enhancing system reliability and troubleshooting.
- Configuring a Maximum Receives count helps prevent endless retry loops by directing unprocessed messages to a DLQ after a set number of attempts.
- Using DLQs improves system resilience by isolating problematic messages from the main workflow.
As a final step, remember it’s always a best practice to clean up resources after completing a lab. Deleting unused resources will help maintain a clutter-free AWS environment.
Happy learning!!!