Lab Details

1. This lab walks you through the steps to create a Cloudtrail and CloudWatch log group , while also creating a metric filter to receive an alarm from CloudWatch via SNS topic.
2. Duration: 1 hour
3. AWS Region: US East (N. Virginia)

Introduction

Cloudwatch

1. AWS Cloudwatch is the service that is used to monitor and collect the metrics from services periodically. This helps provide a clear picture for the users to understand how the resources are performing.
2. It collects data in the form of logs, events and metrics and provides you with an organized view of AWS resources, services and applications that run on AWS.
3. You can use CloudWatch to detect anomalous behavior in your environments and to set alarms, You can visualize data from the logs and take actions to troubleshoot the issue.
4. You can monitor AWS resources such as Amazon EC2, Amazon RDS, Amazon DynamoDB tables, and many others using CloudWatch.
5. You can monitor resource utilization in your account by setting up rules and events to stop or terminate underutilized resources, reducing unnecessary cost.
6. In Autoscaling, servers are stopped or launched based on the events we create in CloudWatch.
7. CloudWatch also offers a feature to store logs for the services running in our account. For example, the logs for lambda functions will be stored within log groups in CloudWatch. Here we can get a detailed error log from any specific function.

CloudTrail

1. AWS CloudTrail is a service that helps us monitor, survey, and audit our AWS Account. 
2. With the help of AWS CloudTrail, the user will be able to log, monitor, and retain account activity associated with actions across the AWS infrastructure. 
3. CloudTrail provides complete account activity of the Amazon Web Services. CloudTrail also manages the functions performed with the help of the AWS Management Console, program line tools, AWS SDKs, and various other AWS services.
4. This event history simplifies security analysis, resource amendment trailing, and troubleshooting.

Tasks

1. Creating a CloudTrail instance and collect the logs in an S3 bucket
2. Creating Log groups to collect the CloudTrail metrics 
3. Creating metric filters to filter on a pattern in the logs
4. Creating SNS topics to be notified when a filter metric triggers an alarm.
5. Testing the above steps by creating an EC2 instance and stopping the instance a couple of times to trigger an alarm.

Architecture Diagram

Lab Steps

Task 1: Launching Lab Environment

1. Launch the lab environment by clicking on . Please wait until the lab environment is provisioned. It will take less than 2 minutes to provision the lab environment.
2. Once the Lab is started, you will be provided with IAM user name, Password, AccessKey and Secret Access Key.
3. Click on the , AWS Management Console will open in a new tab.
4. In the AWS sign in page, the Account ID will be present by default.
   • Leave the Account ID as default. Do not remove or change the Account ID otherwise you cannot proceed with the lab.
5. Copy and paste the IAM user name and Password into AWS Console. Click on Sign in to log into the AWS Console.?

Note : If you face any issues, please go through FAQs and Troubleshooting for Labs.

Task 2: Creating a CloudTrail

1. Make sure to choose the N.Virginia region in the AWS Management console dashboard (present in the top right corner).
2. Navigate and click on CloudTrail, which will be available under the   section of .
3. Click on 
4. Under Create Trail, enter these details:
   • Trail name    : Enter My_cloudtrail
   • Storage Location   :   Create a new S3 Bucket
     • Trail log bucket and folder  :  Leave it as default
   • Log file SSE-KMS encryption  :  Uncheck
   • Additional Settings:
     • Log file validation  :  Uncheck
     • SNS notification delivery  :  Leave it as default

• CloudWatch Logs  :  Check
  • Log group : Leave it as default (i.e New and default log group name)
  • IAM Role : Select New and give the Role name as whiz_role

• Tags: Click Add Tags
  • Key: Enter Name
  • Value: Enter my_logs
• Click on Next.
• Choose Log Events:
  • Leave everything as default and click on Next.
• Review and click on .

5. A CloudTrail instance that delivers logs to an S3 bucket has now been created.

Task 3: Creating Metric Filters for Log Groups in Cloudwatch

1. Click on services and navigate to the  dashboard.
2. Click on  under Logs in the left panel.
3. Click on the log group we just created and click on the Actions.
4. Click on  as shown below:
5. Under Create filter pattern, provide the pattern you need to filter on. For this lab, we are going to filter for stopped instances.

• Filter pattern                 : Enter the pattern { $.eventName= “StopInstances” }
• Select log data to test  : Select the cloudtrail log  in drop-down.

• After completing the above steps, click on .

6. Next we will create a filter using the following details:

• Filter name                           : stoppedInstancecount
• Metric details:
  • Metric namespace  : Enter CloudTrailMetrics
  • Metric name            : Enter EC2stoppedInstanceEventCount
  • Metric value            : Enter 1
  • Default value           : Leave default
• Finally, click on  and review the given details. Click on  to complete the metric filter creation.

Task 4: Creating an Alarm

1. In CloudWatch, select the log group created for our CloudTrail and then click on  at the bottom.
2. Select the Metric filter created in the above step and then click on  as shown below:

3. Specify the metric conditions as follows:

• Namespace : CloudTrailMetrics
• Metric name : EC2stoppedInstanceEventCount
• Statistic        : sum
• Period          : 5 minute
• Conditions:
  • Threshold type                                                 : Static
  • Whenever EC2stoppedInstanceEventCount is :    to 1.
• Click on .

4. Next we’ll configure actions

• Whenever this alarm state is                               : 
• Select an SNS topic                                            : select Create new topic
• Create a new topic                                              : Enter the topic name as My_Ec2count_topic
• Email endpoints that will receive the notification :  Enter your Email address to receive the alert
• Once you provide these details, click on .
  • AWS will send a confirmation email to the Email address provided above. You will needto confirm the email subscription as shown below:

• Click on the  button to complete the alarm creation.

5. Give the name for your alarm and complete the steps as shown below:

• Define a unique name : My_stopped_ec2_alarm
• Alarm description        : Alarm to count the stopped instances count
• Review the details and click on .

6. Navigate to the CloudWatch dashboard and click on alarms. You should see the alarm created in the above step under insufficient data as shown below:

Task 5: Creating an EC2 Instance to Trigger our Alarm

1. Make sure you are in the N.Virginia Region.
2. Navigate to EC2 by clicking on the  menu in the top, then click on  in the  section
3. Switch off the New EC2 experience button present on the left top of menu list. Click on button on the feedback prompt.

4. Navigate to  on the left panel and click on 
5. Search and Choose Amazon Linux AMI: 

6. Choose an Instance Type: Select  and click on 
7. Configure Instance Details: Leave the values as default and click on 
8. Add Storage: Leave the values as default and click on 
9. Add Tags: Click on  

• Key       : Name
• Value    : MyEC2Server
• Click on 

10. Configure Security Group:

• Name            : My EC2 SecurityGroup 
• Description    : Security group for my EC2 server
• To add SSH:
  • Choose Type: SSH
  • Source: Anywhere

11. Review and Launch : Review all settings and click on .
12. Key Pair : Create a new key Pair and click on  . After that, click on .
13. Launch Status:Your instance is now launching, Click on the instance ID and wait for complete initialization of instance.

14. Once the EC2 instance launches successfully, start and stop the instance 2 to  3 times as shown in the below screenshot:

15. Navigate to the CloudWatch console to see the newly-created alarm. It should show the state as insufficient data, as shown below:

16. Check the email that you provided to SNS for an email looking like the one below:

Task 6: Validation Test

1. Once the lab steps are completed, please click on the  button on the right side panel.
2. This will validate the resources in the AWS account and displays whether you have completed this lab successfully or not.
3. Sample output : 

Completion and Conclusion

1. You have successfully created a CloudTrail and an S3 bucket to store logs.
2. You have created CloudWatch log groups and a metric filter for stopped EC2 instances.
3. You have successfully created SNS topic to receive the alert from CloudWatch.
4. You have successfully launched an EC2 instance.
5. You successfully stopped the instance a few times to check the working of the alarm.