Review Set 2 – Question 1

[SalientListener]

Category: CSAP – Accelerate Workload Migration and Modernization

A company has several NFS shares in its on-premises data center that contains millions of small log files totaling around 50TB in size. The files in these NFS shares need to be migrated to an Amazon S3 bucket. To start the migration process, the solutions architect requested an AWS Snowball Edge device that will be used to transfer the files to Amazon S3. A file interface was configured on the Snowball Edge device and is connected to the corporate network. The Solutions Architect initiated the snowball cp command to start the copying process, however, the copying of data is significantly slower than expected.

Which of the following options are the likely cause of the slow transfer speed and the recommended solution?

(A)This is due to encryption overhead when copying files to the Snowball Edge device. Open multiple sessions to the Snowball Edge device and initiate parallel copy jobs to improve the overall copying throughput.

(B)The file interface of the Snowball Edge has reached its throughput limit. Change the interface to an S3 Adapter instead for a significantly faster transfer speed.

While I agree (A) is the right Answer based on AWS recommendation – Stephen Maarek’s course suggests using (B) given S3 Adapter provides a much higher throughput of 250-400MBs compared to regular file interface of 25-50MBPs. Based on this 2021 AWS documentation , regular NFS file interface speeds now provide the SAME speeds as S3 adapters. But nowhere does it mention the 100GPS you mention in your answer explaining why B is wrong. https://aws.amazon.com/about-aws/whats-new/2021/08/aws-snowball-edge-storage-optimized-nfs-data-transfer/#:~:text=After%20mounting%20the%20file%20share,file%20interface%20on%20SBE%20devices.

Question is:

1. Can you please clarify where you got this 100GBps figure from in explaining the incorrect (B) answer?

2. And when is a good use case to use S3 adapters based on my research above? The AWS documentation is useless as they dont explain when should we use S3 adapters esp given the 2021 link above.

• This discussion was modified 3 months, 2 weeks ago by  SalientListener.

[Neil-TutorialsDojo]

Hi SalientListener,

Good day! Thank you for your feedback.

For your first question, I think there is a misunderstanding about the explanation of option B you mentioned. Upon checking, there is no mentioned “100GBps” in the explanation why option B (“The file interface of the Snowball Edge has reached its throughput limit. Change the interface to an S3 Adapter instead for a significantly faster transfer speed”) is incorrect. Kindly see the image attached.

To clarify why this option is incorrect, as mentioned int the explanation, the file interface of the Snowball Edge device is not the limiting factor in the scenario. Instead, the issue is the encryption overhead when copying files to the device. To improve the overall copying throughput, you should open multiple sessions and initiate parallel copy jobs. This approach will maximize the network capabilities of the Snowball Edge device. Therefore, the best option here is Option A (This is due to encryption overhead when copying files to the Snowball Edge device. Open multiple sessions to the Snowball Edge device and initiate parallel copy jobs to improve the overall copying throughput).

As for your second question, the S3 Adapter is best used when transferring large amounts of small files efficiently. It maximizes throughput, typically achieving speeds between 250-400 MB/s, making it ideal for applications requiring faster speeds than the file interface’s typical performance. Additionally, the S3 Adapter simplifies data management by enabling direct uploads to an S3 bucket, reducing the complexity of handling numerous small files. Hence, it is also well-suited for scenarios where minimizing operational overhead is a priority.

I do understand that the scenario and question, wasn’t able to clearly mention the requirements. And with that said, I apologize for any confusion this may have cause. Rest assured that we will do the necessary update as soon as we can.

If you have further clarifications, pls let us know.

Regards,
Neil @ Tutorials Dojo

[SalientListener]

Hi Neil

• Im still not clear why S3 Adapter is not correct – it does satisfy all the requirements for an efficient transfer (large no of files, extremely fat rates of transfer 250-400Mbps etc.) What is the clear distinction that favors writing in parallel vs using S3 Adapter for the same millions of files scenario in the questions? If posted the same question to Amazon Q AI bot and it (and chatgpt) both of which gave this answer “Writing to a Snowball device using the S3 Adapter can potentially be faster than using the regular file interface: However, the actual performance difference between using the S3 Adapter versus the regular file interface can vary depending on factors such as the size and type of data being transferred, the network conditions, and the specific workload requirements.” -> without additional details in the question – I feel there’s no BLACK or WHITE difference in your explanation as to why you favor File interface in parallel vs S3 Adapter?
• The “100 GBPs” is in the explanation of the final incorrect answer – this is where I was looking to understand where you pulled this info from?

The option that says: The file interface of the Snowball Edge is limited by the network interface speed. Connect the device directly using a high-speed USB 3.0 interface instead to maximize the copying throughput is incorrect. Although some revisions of USB 3.0 or USB 3.1 can support up to 5 Gbps to 10 Gbps speeds, the network interface on the Snowball Edge supports up to 100 Gbps. You can maximize throughput by issuing multiple copy commands to the Snowball device. 

[Neil-TutorialsDojo]

Hi SalientListener,

You have a point there. The distinction should revolve around the utilization of network capabilities and encryption overhead. However, as I mentioned in my previous response, the requirements weren’t clearly mentioned. We will update this as soon as we can to avoid any more confusion. You are right in your observation.

Regarding the 100 Gbps figure, thank you for clarifying that. I believe it was pulled from these documentations:

• https://docs.aws.amazon.com/snowball/latest/developer-guide/whatisedge.html#edge-feature-overview
• https://aws.amazon.com/snowball/features/

Kindly check the images attached for more details.

Regards,
Neil @ Tutorials Dojo