-
Confusing question about Glue & Quicksight
updated 4 months, 2 weeks ago
2 Members
·
2
Posts
-
The question: “An organization collects daily user interaction and app performance logs from its mobile app. These initially raw logs are transformed by AWS Glue for Spark Jobs and then stored in an Amazon S3 bucket for visualization with Amazon QuickSight.
The company’s Data Engineer has noticed a gradual slowdown in the QuickSight Dashboard’s query performance over time. Upon investigation, the engineer determined that the long duration of the AWS Glue Jobs is the root cause.
What combination of actions should the Data Engineer consider to improve query performance?”
-The question says that the data is “stored in an Amazon S3 bucket for visualization with Amazon QuickSight.”, so presumably Quicksight is reading the data directly from S3. If so, how could it also be that “the long duration of the AWS Glue Jobs is the root cause.”? So the premise here is confusing right from the start.
-Furthermore, the correct answers to the questions are said to be:
<ul data-question_id=”14075″ data-type=”multiple”>
- Modify the AWS Glue Job worker type configuration to a larger instance type.
- Arrange the data within the S3 bucket into partitions by year, month, and day.
One of these affects the Glue Job, the other the way the data is stored. How is a Quicksight Report performance affected by a Glue job worker type? It could be that the data would be more readily available (refreshed faster) if the Worker Type is larger but as far as I know, Quicksight doesn’t “kick off” a Glue Job when a Report is run.
Perhaps I’m missing something?
-
Hi or-3,
Thank you for posting here.
Amazon QuickSight uses the cleaned and structured data prepared by AWS Glue for visualizations and dashboards. By upgrading the Glue instance size and partitioning the data, you not
only improve the efficiency of the data processing jobs but also enhance
the responsiveness and performance of the analytics and dashboards in
QuickSight.
I hope this helps.Regards,
Neil @ Tutorials Dojo
Log in to reply.