Review Set 4 – Q54 – lambda scaling?

[Yoj]

Hello

Question is :

A top Internet of Things (IoT) company has developed a wrist-worn activity tracker for soldiers deployed in the field. The device acts as a sensor to monitor the health and vital statistics of the wearer. It is expected that there would be thousands of devices that will send data to the server every minute and after 5 years, the number will increase to tens of thousands. One of the requirements is that the application should be able to accept the incoming data, run it through ETL to store in a data warehouse, and archive the old data. The officers in the military headquarters should have a real-time dashboard to view the sensor data.

Response is :

Send the raw data directly to Amazon Kinesis Data Firehose for processing and output the data to an S3 bucket. For archiving, create a lifecycle policy from S3 to Glacier. Use Lambda to process the data through Amazon EMR and sends the output to Amazon Redshift.

With that architecture, does lambda not suppose to be a bottleneck?

Question is about to ten of thousands of devices sending items to S3 every 5min

Lambda can’t scale to ten of thousands of concurrent execution in a given region when objects are pushed in bucket S3? This is why i have rejected this answer?

[Neil-TutorialsDojo]

Hello Yoj,

Good day! Thank you for your feedback. You raise a valid point about the scalability of AWS Lambda when
processing a large number of objects pushed to an S3 bucket. While
Lambda is designed to scale automatically, there are some considerations
to keep in mind like initial burst of traffic.

“For an
initial burst of traffic, the cumulative concurrency in a Region can
reach between 500 and 3000 instances per minute, depending on the
Region.”
-https://docs.aws.amazon.com/lambda/latest/operatorguide/scaling-concurrency.html

With that said, I appreciate your detailed observation. Rest assured that we will use your input to make our content better. Thank you once again for helping us improve our resources.

Regards,
Neil @ Tutorials Dojo