Ends in
00
days
00
hrs
00
mins
00
secs
SHOP NOW

🎆NEW YEAR SALE EXTENSION - Practice exams as low as $9.74 USD & eBooks as low as $2.99 USD

Guided Lab: Processing Amazon DynamoDB Streams with AWS Lambda

Description

Amazon DynamoDB streams is a feature that captures data modification events in DynamoDB tables in real-time. It provides a time-ordered sequence of item-level changes in a table, allowing you to track and respond to these changes as they occur. This powerful functionality opens up a world of possibilities for building reactive, event-driven applications and implementing complex data processing workflows.

At its core, DynamoDB streams create a separate endpoint containing a time-ordered sequence of item-level modifications made to a DynamoDB table. Each stream record includes information on a single modification to the data in the table. When enabled, DynamoDB streams captures a “before” and “after” image of every modified item, giving you a comprehensive view of how your data is changing over time.

In this lab, you’ll gain hands-on experience in setting up and working with DynamoDB Streams. You’ll learn how to enable streams on a DynamoDB table, configure stream settings, and explore ways to process and utilize the stream data with the integration of an AWS Lambda function. By the end of this session, you’ll have a solid understanding of how to leverage DynamoDB Streams to build more responsive, scalable, and robust applications on AWS.

Prerequisites

This lab assumes that you have a basic understanding and concepts of Amazon DynamoDB and AWS Lambda function.

If you find any gaps in your knowledge, consider taking the following lab:

Objectives

By the end of this lab, you will:

  • Configure and enable DynamoDB Streams on an existing DynamoDB table.
  • Create and configure a Lambda function to process events from DynamoDB Streams.
  • Test and verify the successful capture and processing of data changes in real-time using DynamoDB Streams.
  • Understand the basic DynamoDB Streams events: Insert, Modify, and Remove

Lab Steps

Enabling an Amazon DynamoDB stream

1. Navigate to Amazon DynamoDB

2. Click “Create table” to create a new DynamoDB table

3. Enter the following configuration for the Table details:

  • Table name: This could be anything relevant to the data you’re storing. For example, we will put “Foods”.
  • Partition key: This could be any attribute of your data. For example, let’s put “Desserts”. Also, don’t change its data type on the right side as it should remain as “String”
  • Leave the other configurations as default and click Create Table at the bottom of the page.

4. Select the DynamoDB table that was created.

5. Go to the Exports and Streams section, scroll down to ‘DynamoDB stream details’ then click “Turn on

6. On the DynamoDB stream details, tick “New and old images” then click “Turn on stream

Creating an AWS Lambda function to process DynamoDB streams

1. After the DynamoDB stream was turned on successfully, navigate to AWS Lambda

2. Click “Create new function“ and enter the following configuration for the Lambda function:

  • Author from Scratch
  • Function name: ProcessDynamoDBStream
  • Runtime: Python 3.13
  • Change default execution role: Use an existing role → PlayCloud-Sandbox
  • Click Create function at the bottom of the page.

3. Go to its Code Monitor and enter the following code, then click Deploy:

import json
def lambda_handler(event, context):
    for record in event['Records']:
        if record['eventName'] == 'INSERT':
            print(f"New item added: {record['dynamodb']['NewImage']}")
        elif record['eventName'] == 'MODIFY':
            print(f"Item modified: {record['dynamodb']['NewImage']}")
        elif record['eventName'] == 'REMOVE':
            print(f"Item removed: {record['dynamodb']['OldImage']}")
    return {
        'statusCode': 200,
        'body': json.dumps('Processed event successfully')
    }

Info: This code will process the changes that occurred in the DynamoDB table, which are considered as stream events. These three DynamoDB stream events are Insert, Modify, and Remove. After that, it will return a success status (200) with a confirmation message and log the processed event that can be viewed on CloudWatch logs. Stated below are short description of the three DynamoDB stream events:

  • INSERT: Logs the newly added item using NewImage.
  • MODIFY: Logs the updated or changed item using NewImage.
  • REMOVE: Logs the deleted item using OldImage.

4. Scroll up to Function Overview, then click Add Trigger

5. Enter the following Trigger Configuration:

  • Trigger Configuration: DynamoDB
  • DynamoDB table: Food
  • Batch Size: 1
  • Leave the other configurations as default and click Add at the bottom of the page.

Testing the setup for processing the DynamoDB stream

1. To test the Setup, return to your created DynamoDB table, click Actions → Create item.

2. Enter any random ‘dessert’ or string on the Empty Value then click Create item.

3. Go back to the ‘ProcessDynamoDBStream’ Lambda function, go to Monitor section, then click “View CloudWatch logs

4. Click the log stream at the bottom to view it. You can see on the displayed logs that the item you created has now been added to the DynamoDB table.

5. You can also try adding, removing, and modifying more data in your created DynamoDB table, then check here in the CloudWatch logs to see if the changes you made were reflected.

Congratulations! You have successfully completed the lab on processing Amazon DynamoDB Streams with AWS Lambda. Understanding DynamoDB Streams and its real-time event processing capabilities is crucial for designing scalable and event-driven applications. By being introduced to fundamental DynamoDB stream events such as Insert, Modify, and Remove, you can now achieve workflow automation, which helps you maintain data consistency across systems. This knowledge is essential for developing applications that need real-time data processing and seamless integration with other AWS services.

Skip to content