AWS Lambda & EventBridge | Schedule Start And Stop  Of EC2 Instances Based On Tags...

AWS Lambda & EventBridge | Schedule Start And Stop Of EC2 Instances Based On Tags...

In this blog we have to start and stop EC2 instances based on tags during working hours to save cost. We are going to automate this by using python lambda function and schedule it based on the working hours.

Prerequisite

An AWS Account An IAM User with:

  • AWS Management Console access to verify your EC2 instances launched,listed and terminated.
  • The IAM permissions required to perform IAM, EC2, and CloudWatch activities. IAM policy creation and AWS Application Programming Interface (API) permissions are outside this article’s scope. Always adhere to the principle of least privilege when authorizing accounts to perform actions. Administrative access to an EC2 Instance.

Create new role for our lambda functions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:StartInstances",
                "ec2:DescribeTags",
                "logs:*",
                "ec2:DescribeInstanceTypes",
                "ec2:StopInstances",
                "ec2:DescribeInstanceStatus"
            ],
            "Resource": "*"
        }
    ]
}

Launch AWS EC2 Instance using python script

  1. Python code in one module gains access to the code in another module by the process of importing it. The import statement combines two operations it searches for the named module, then it binds the results of that search to a name in the local scope.
    import boto3
    
  2. We will invoke the client for EC2
    client = boto3.client('ec2')
    
  3. To launch EC2 instances we have to use method "run_instances()". This method helps us launch AWS EC2 instances based on our requirement.
    response =client.run_instances(<arguments>)
    
  4. Goto link where you will find all arguments list. Based on your requirement you can put this arguments to launch your EC2 instances. This document also mentions datatype of the parameter.
    Note:- Arguments which are with "REQUIRED" tags mentioned in documentation is mandatory, if you don't specify those arguments code block to launch EC2 will not execute successfully.
    Example:- "MinCount", "MaxCount".
    Below code will launch EC2 instance based on your provided input.
    resp=client.run_instances(ImageId='ami-0742b4e673072066f',
                           InstanceType='t2.micro',
                           MinCount=2,
                           MaxCount=2,
                           KeyName='CustomVPC',
                           TagSpecifications=[
                               {
                                   'ResourceType': 'instance',
                                   'Tags': [{'Key': 'Name','Value': 'Linux Server'},
                                            {'Key': 'Env','Value': 'Dev'}]
                               },
                           ],
                           )
    
  5. Once above method will run it will launch EC2 and launched EC2 information will be captured in variable "resp". It will return infomation in dictonary, so "resp" would be a dictonary.
  6. Now we will traverse the dict using for loop to print list of instances launched by "run_instances" method.
    for i in resp['Instances']:
    print("Instance ID Created is :{} Instance Type Created is : {}" .format(i['InstanceId'],i['InstanceType']))
    
    Entire Code is listed below
    import boto3
    client = boto3.client('ec2')
    resp=client.run_instances(ImageId='ami-0742b4e673072066f',
                          InstanceType='t2.micro',
                          MinCount=2,
                          MaxCount=2,
                          KeyName=<Your Keyname>,
                          TagSpecifications=[
                              {
                                  'ResourceType': 'instance',
                                  'Tags': [{'Key': 'Name','Value': 'Linux Server'},
                                           {'Key': 'Env','Value': 'Dev'}]
                              },
                          ],
                          )
    for i in resp['Instances']:
    print("Instance ID Created is :{} Instance Type Created is : {}" .format(i['InstanceId'],i['InstanceType']))
    

Create Lambda Function To Stop EC2 Instances Based On Tags

  1. Goto Lambda console and click on create function image.png
  2. Select "Author From Scratch" , Function name = Lambda_Start, Runtime= Python and role we created with above policy attached to this blog and click on create function. image.png
  3. Goto code editor and start writing the code. image.png
  4. We will import boto3 and json modules.
    import json
    import boto3
    
  5. We will invoke resource for EC2 along with the region name where we are going to perform this activity.
    ec2 = boto3.resource('ec2', region_name='us-east-1')
    
  6. Now we will first search for instance is which are in running state and are having Env tag as "Dev" by using instances.filter(). Checkout he official documentation for this method here
    def lambda_handler(event, context):
     instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
    
  7. Now we will traverse the list returned using for loop and fetch instance ids and save it in variable id.
    def lambda_handler(event, context):
    instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
    for instance in instances:
        id=instance.id
    
  8. Stopping instances given a list of instance IDs uses Boto3 collection filtering. This will stop all running instances having Env tag with value as Dev.
    def lambda_handler(event, context):
     instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
     for instance in instances:
         id=instance.id
         ec2.instances.filter(InstanceIds=[id]).stop()
         print("Instance ID is stopped:- "+instance.id)
     return "success"
    
    To view entire github code please click here

Create Lambda Function To Start EC2 Instances Based On Tags

  1. Goto Lambda console and click on create function image.png
  2. Select "Author From Scratch" , Function name = Lambda_Stop, Runtime= Python and role we created with above policy attached to this blog and click on create function. image.png
  3. Goto code editor and start writing the code. image.png
  4. We will import boto3 and json modules.
    import json
    import boto3
    
  5. We will invoke resource for EC2 along with the region name where we are going to perform this activity.
    ec2 = boto3.resource('ec2', region_name='us-east-1')
    
  6. Now we will first search for instance is which are in running state and are having Env tag as "Dev" by using instances.filter(). Checkout he official documentation for this method here
    def lambda_handler(event, context):
     instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
    
  7. Now we will traverse the list returned using for loop and fetch instance ids and save it in variable id.
    def lambda_handler(event, context):
    instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
    for instance in instances:
        id=instance.id
    
  8. Starting instances given a list of instance IDs uses Boto3 collection filtering. This will stop all running instances having Env tag with value as Dev.
    def lambda_handler(event, context):
     instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']},{'Name': 'tag:Env','Values':['Dev']}])
     for instance in instances:
         id=instance.id
         ec2.instances.filter(InstanceIds=[id]).start()
         print("Instance ID is started :- "+instance.id)
     return "success"
    
    To view entire github code please click here

Schedule Lambda To Start EC2

  1. Open Amazon Eventbridge service and open rules. And click on create rule image.png
  2. Now we will create rule and schedule it. For scheduling you we will have to use cron expression as displayed below. You can find official documentation here image.png
  3. Now lets create our first rule to start EC2 instance. First we will define name and description as below image.png
  4. Now we will define Cron job expression where we will define that this job should run on 11 am IST and from monday to friday . Expression would be
    0 11 ? * 2-6 *
    
    image.png
  5. Select target as the lambda function and select our lambda function to start EC2 instance and click on create. image.png

Schedule Lambda To Stop EC2

  1. Open Amazon Eventbridge service and open rules. And click on create rule image.png
  2. Now we will create rule and schedule it. For scheduling you we will have to use cron expression as displayed below. You can find official documentation here image.png
  3. Now lets create our rule to stop EC2 instance. First we will define name and description as below image.png
  4. Now we will define Cron job expression where we will define that this job should run on 11 am IST and from monday to friday . Expression would be
    0 20 ? * 2-6 *
    
    image.png
  5. Select target as the lambda function and select our lambda function to stop EC2 instance and click on create. image.png image.png

Youtube Tutorial

Resource clean up

  • Terminate EC2 instances
  • Delete Lambda functions for start and stop
  • Delete Eventbridge rules for lambda functions
  • Delete Lambda Role

Conclusion

Boto3 provided inbuild methods for AWS resources using which many task can be automated by writing a python script.

Stay tuned for my next blog.....

So, did you find my content helpful? If you did or like my other content, feel free to buy me a coffee. Thanks.

Did you find this article valuable?

Support Dheeraj Choudhary by becoming a sponsor. Any amount is appreciated!