Guided Lab: Capture network traffic information with VPC flow logs to Amazon S3 Bucket
Description
Monitoring the traffic flowing through your VPC is essential for analyzing security and performance. AWS VPC Flow Logs can capture details about IP traffic going to and from your network interfaces. You can publish these logs to Amazon S3 for further analysis or archiving.
In this lab, you will learn how to create a VPC Flow Log that captures network traffic information and publishes it to Amazon S3 for long-term storage.
Prerequisites
This lab assumes basic knowledge of AWS networks and core services such as EC2, S3, and VPC.
If you’re unfamiliar with these services, consider exploring the following resources:
- Creating an Amazon EC2 instance (Linux)
- Creating a Custom Virtual Private Cloud (VPC) from scratch
- Guided Lab: Creating an Amazon S3 bucket
- Capture network traffic information with VPC flow logs to CloudWatch Logs
Objectives
By the end of this lab, participants will be able to:
- Set up an Amazon S3 bucket to store VPC Flow Log data.
- Create and configure VPC Flow Logs to capture IP traffic and publish logs to Amazon S3.
- Verify the flow logs by generating and reviewing network traffic data stored in S3.
Subscribe to access AWS PlayCloud LabsLab Steps
Set up an Amazon S3 Bucket
1. Navigate to the S3 Console:
2. Create a New Bucket:
-
Click the “Create bucket” button. Fill in the following details:
- Bucket Name:myflowlogsbucket3000
- Click Create Bucket to finalize the setup.
3. Take note of the Amazon Resource Name (ARN) of your S3 bucket.
Launch an EC2 Instance
1. Navigate to the EC2 Dashboard
2. Launch an EC2 Instance using the following configurations:
- Name: MyWebServer
- AMI: Amazon Linux
- Instance type: t2.micro
- Key pair: (Please create a new one.)
- Key pair name: myKeyPair
- Key pair type: RSA
- Private key file format: .pem
- Network settings: (Click “Create security group”)
- Auto-assign public IP: Select Enable
- Firewall (security groups): tick on the Create security group
- Ensure that Allow SSH traffic from is checked and is My IP
- Click Launch Instance.
3. Wait for the EC2 instance to be in the Running state.
4. Select the instance and click on the Actions dropdown.
5. Navigate on Networking > Change source/destination check
6. Tick the stop checkbox and Save
The Source/Destination Check in an EC2 instance is a network setting that controls whether the instance must be the source or destination of traffic it sends or receives.
- Enabled (default): The instance only accepts traffic addressed to its own IP. This is when Stop checkbox is unchecked.
- Disabled: The instance can forward traffic, useful for NAT, routing, or firewall role. This is when Stop checkbox is checked.
Create a VPC Flow Log for S3
1. Navigate to the VPC Dashboard.
2. In the left navigation pane, click VPC and select the VPC where your EC2 instance is running. In this lab, we choose the default VPC.
3. Create a Flow Log:
- Click Actions and select Create flow log.
- Name: my-flow-log-to-s3
- Filter: AllMaximum Aggregation Interval: 1 minute
- Destination: Select Send to an S3 bucket.
- S3 Bucket ARN: Enter the ARN for your S3 bucket, e.g., arn:aws:s3:::myflowlogsbucket3000.
4. Click Create Flow Log to complete the configuration.
Generate Network Traffic and Verify Flow Logs
1. To generate network traffic, SSH into your EC2 instance and run the ping command to any website. For example:
ping google.com
2. Review the Logs in S3:
- Navigate to your bucket and check for the presence of log files. The logs will be stored in a path similar to the image below.
3. Download one of the files and check.
Congratulations! You have successfully created a VPC Flow Log that publishes logs to Amazon S3. With this setup, you can retain logs long-term for compliance, auditing, or in-depth analysis. This solution provides visibility into your VPC traffic while leveraging S3’s scalability and durability for storing large amounts of data.