Streaming one DynamoDB database changes to another DynamoDB database

[vadym]

Hello,

I’m studying the course and I encountered a quiz question which asks to stream one DynamoDB database changes to another DynamoDB database. It has various options one of which is using Kinesis data stream to stream changes and another is to use DynamoDB streams.

Explanation is

“Using the Amazon Kinesis Adapter is the recommended way to consume streams from Amazon DynamoDB. Since DynamoDB Streams API is intentionally similar to that of Kinesis Data Streams, a service for real-time processing of streaming data at a massive scale, there’s no need to use Amazon Kinesis.”.

Could you please explain why we shouldn’t use Amazon Kinesis Streams if that’s the recommended way? I get that DynamoDB Streams are similar but they aren’t a recommended way!

I also had a similar question earlier (which I don’t remember) about running Lambda using DynamoDB changes as an input which had an opposite answer i. e. Kinesis Streams were the correct answer and DynamoDB Streams were considered incorrect.

I’m confused.

[Carlo-TutorialsDojo]

Hello vadym,

Thank you for your feedback.

The Kinesis Data Stream support for DynamoDB was recently added when this question was created. So by default, I made sure that all answers related to Amazon Kinesis are automatically crossed out. However, I admit that the rationale that I provided was poor and somewhat confusing.

Let me clear this one out:

“Using the Amazon Kinesis Adapter is the recommended way to consume streams from Amazon DynamoDB. Since DynamoDB Streams API is intentionally similar to that of Kinesis Data Streams, a service for real-time processing of streaming data at a massive scale, there’s no need to use Amazon Kinesis.”.

>> This statement refers to the “processing” of records from a stream (can be DynamoDB stream or Kinesis Data Stream) which can be done via KCL or DynamoDB Streams API. It does not pertain to the “capturing” of changes.

To stream changes from one DynamoDB table to another, you must:

1. Enable DynamoDB Stream or Kinesis Data Stream.

2. Poll the changes from the stream using a Lambda function or a KCL application.

We will revise this item so it wouldn’t cause confusion for others as well.

Let me know if you have any other queries.

Regards,

Carlo @ Tutorials Dojo

[vadym]

Understood. Thanks a lot!