Wording in scenario confusing (dynamodb/sqs/high number of write operations)

[kung]

Scenario: donation website with high number of write operations

I’m very well aware of using an SQS queue for decoupling, but I found the answer wording a bit misleading (so I picked ‘the wrong’ option).

“Use an SQS queue to buffer the large incoming traffic to your Auto Scaled EC2 instances, which processes and writes the data to DynamoDB.”

This gave me the impression that

a. you were not using cloudfront (as that was only mentioned in another answer),
b. your webserver (apache/nginx) would be running on the mentioned EC2 instances

as you are mentioning ‘large incoming traffic’ , which I interpret as web traffic (but how to ‘buffer’ that?).

This would give the weird setup of webtraffic coming to EC2, which drops a message in SQS, and then picks it out of the SQS queue again to write it to DynamoDB. (of course auto scale it for multiple EC2’s).

Giving that this sounded a bit weird, I chose the option with CloudFront, knowing that DynamoDB can scale a lot anyway (so not to worry about that to much).

What do you think?
Cheers,
Robert

[TutorialsDojo-Support]

Hi Robert,

Thanks for sharing your feedback.

The question did not specifically say it is using EC2 instances with Apache/Nginx as the front-end for the incoming requests. It could be a separate setup, either a static page on S3 or another cluster of EC2 instances.

Either way, the question would like to find the best solution to scale and accommodate the incoming request to write to the database (in the order of millions).

“These features of the donation website will eventually result in a high number of write operations on their database tier considering that there are millions of generous donors around the globe who want to help. “

Yes, DynamoDB can scale and CloudFront can cache user requests, but DynamoDB has a limit for its scaling. SQS on the other hand can be store virtually an unlimited number of messages. So millions of requests can be temporarily stored while the consumer EC2 instances process each request and save it into the DynamoDB table. This reduces overhead for scaling IOPS for your DynamodDB, which is also expensive.

Regards,

Kenneth Samonte @ Tutorials Dojo