second Amazon Kinesis stream

[VG-AWS]

17. QUESTION

Category: CSAP – Continuous Improvement for Existing Solutions

A leading mobile game company has an application running on Elastic Beanstalk that continuously collects player-game interactions and player’s behavior then feeds the data into an Amazon Kinesis stream. A second Elastic Beanstalk app generates key performance indicators (KPIs) into a DynamoDB table and powers the game leaderboard. After a few weeks, there has been a technical problem in the Kinesis data stream which resulted in data loss for your application.

Which of the following is the most efficient and most scalable option to prevent any data loss for this application?

Why it can’t be Launch a second Amazon Kinesis stream in another Availability Zone then use Data Pipeline to replicate data across Kinesis streams.

[TutorialsDojo-Support]

Hi vipul-gupta,

Thank you for your feedback.

This question asks for “the most efficient and most scalable option” in this scenario.

“Why it can’t be Launch a second Amazon Kinesis stream in another Availability Zone then use Data Pipeline to replicate data across Kinesis streams.”

This option is not the most efficient because you are essentially duplicating the data the is already there, this also has additional cost. Plus this option heavily relies on the first Kinesis Data Stream, if it suddenly fails, then data on the stream is lost, replication will not work too.

AWS provides tools to be used for these types of scenarios such as Amazon Kinesis S3 connector or Amazon Kinesis Data Firehose that can archive the data from Kinesis data stream into an S3 bucket. If the Kinesis stream fails, you will have your data stored on S3.

Hope this helps.

Regards,

Kenneth Samonte @ Tutorials Dojo