Dheeraj Choudhary
Dheeraj Choudhary's Blog

Dheeraj Choudhary's Blog

Schedule Automatic Detection Of Non Associated AWS Elastic IP's In AWS Account On Weekly Basis And Notify | AWS Cloudformation & AWS CLI

Schedule Automatic Detection Of Non Associated AWS Elastic IP's In AWS Account On Weekly Basis And Notify | AWS Cloudformation & AWS CLI

Dheeraj Choudhary
·May 9, 2022·

11 min read

Subscribe to my newsletter and never miss my upcoming articles

Play this article

Table of contents

Infrastructure as Code (IaC) is the managing and provisioning of infrastructure through code instead of through manual processes. With IaC, configuration files are created that contain your infrastructure specifications, which makes it easier to edit and distribute configurations.

This yaml template is created for DevOps - Infrastructure Automation on AWS Project. By using this template we will Schedule Automatic Detection Of Non Associated AWS Elastic IP's In AWS Account On Weekly Basis And Notify, buckle up 🚴‍♂️ and lets get started and understand core cloudformation concepts by implementing it...🎬

❗️❗️❗️ Pre-Requisite ❗️❗️❗️

1️⃣ Add visual studio code extension [Mandatory]

2️⃣ Adding VS Code Indentation Extension For Cloudformation Templates [Optional]

3️⃣ Deploy VPC, IGW & Associate [Mandatory]

4️⃣ Deploy only public subnet template from below blog [Mandatory].Make sure to create 2 public subnets

Parameters:  
  CustomVPC:
    Description: Select One VPC available in your existing account
    Type: AWS::EC2::VPC::Id
    Default: <"Your VPC ID">
  CustomInternetGateway:
    Description: Select One internet gateway available in your existing account
    Type: String
    Default: <"Your Internet Gateway ID">

Resources: 
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: "us-east-1a"
      MapPublicIpOnLaunch: true
      VpcId: !Ref CustomVPC
      CidrBlock: 10.0.0.0/25
      Tags:
        - Key: Name
          Value: PublicSubnet1
  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: "us-east-1b"
      MapPublicIpOnLaunch: true
      VpcId: !Ref CustomVPC
      CidrBlock: 10.0.0.128/25

      Tags:
        - Key: Name
          Value: PublicSubnet2
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref CustomVPC
      Tags:
        - Key: Name
          Value: PublicRouteTable
  PublicRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref CustomInternetGateway
  PublicSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet1
      RouteTableId: !Ref PublicRouteTable
  PublicSubnetRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet2
      RouteTableId: !Ref PublicRouteTable
Outputs:
  outputPublicSubnets1:
    Description: A reference to the created Public subnet
    Value: !Ref PublicSubnet1
    Export: 
      Name: PublicSubnet1
  outputPublicSubnets2:
    Description: A reference to the created Public subnet
    Value: !Ref PublicSubnet2
    Export: 
      Name: PublicSubnet2

🔊 To view double subnet github code click here

5️⃣ EC2 instance with Security group [Reference]

🎨 Diagrammatic Representation 🎨

image.png

Template Components Planning Before Build

2.png

🔳 Parameters

CustomVPC :- Using this parameter for VPC "AWS::EC2::VPC::Id" we can list existing VPC list into the account and select anyone from them. Apart from this list we can also you default value if no value is selected in the parameter.
PublicSubnet: Using this parameter for Subnet "List" we can list existing subnet list from the account and select anyone from them. Apart from this list we can also you default value if no value is selected in the parameter.

Parameters:
  CustomVPC:
    Description: Select One VPC available in your existing account
    Type: AWS::EC2::VPC::Id
    Default: "<your default VPC ID>"
  PublicSubnet1:
    Description: Select one public subnet available in your existing account
    Type: AWS::EC2::Subnet::Id
    Default: "<your default public subnet id>"
  PublicSubnet2:
    Description: Select one public subnet available in your existing account
    Type: AWS::EC2::Subnet::Id
    Default: "<your default public subnet id>"

🔳 Resources

VPCSecurityGroup:- We are going to allow all traffic
GroupName:- This property is used to mention security group name.
GroupDescription:- This property is used to mention security group description and its mandatory property for this resource.
SecurityGroupIngress:- This property is used to add ingress rules for [udp/tcp] ports enabled secured access to your resources.
Tags:- One of the most important property used in all resources. Always make sure to attach tags for all your resources.

  VPCSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: !Ref CustomVPC
      GroupName: "AllowVPCTraffic"
      GroupDescription: "Allow All Traffic"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 0
          ToPort: 65535
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: 0
          ToPort: 65535
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: VPCSecurityGroup

LambdaSecurityGroup:- We are going to allow all traffic
GroupName:- This property is used to mention security group name.
GroupDescription:- This property is used to mention security group description and its mandatory property for this resource.
SecurityGroupIngress:- This property is used to add ingress rules for [udp/tcp] ports enabled secured access to your resources.
Tags:- One of the most important property used in all resources. Always make sure to attach tags for all your resources.

  LambdaSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: !Ref CustomVPC
      GroupName: "AllowEc2Traffic"
      GroupDescription: "Allow all traffic"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 0
          ToPort: 65535
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: 0
          ToPort: 65535
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: LambdaSecurityGroup

Note:- If you are launching it in your default AWS Account VPC no need to create VPC endpoint.

Endpoints:- An endpoint enables you to create a private connection between your VPC and the service. The service may be provided by AWS, an AWS Marketplace Partner, or another AWS account.
ServiceName:- Specify the name of the service endpoint you want to provision.
VpcId:- The ID of the VPC in which the endpoint will be used.
SubnetIds:- The ID of the subnets in which to create an endpoint network interface. You must specify this property for an interface endpoints or a Gateway Load Balancer endpoint. You can't specify this property for a gateway endpoint. For a Gateway Load Balancer endpoint, you can specify only one subnet.
SecurityGroupIds:- The IDs of the security groups to associate with the endpoint network interface. Security groups are supported only for interface endpoints.

  Ec2Endpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      ServiceName: !Sub 'com.amazonaws.${AWS::Region}.ec2'
      VpcId: !Ref CustomVPC
      SubnetIds: 
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroupIds:
        - !Ref VPCSecurityGroup
  EventEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      ServiceName: !Sub 'com.amazonaws.${AWS::Region}.event'
      VpcId: !Ref CustomVPC
      SubnetIds: 
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroupIds:
        - !Ref VPCSecurityGroup  
  SnsEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      ServiceName: !Sub 'com.amazonaws.${AWS::Region}.sns'
      VpcId: !Ref CustomVPC
      SubnetIds: 
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroupIds:
        - !Ref VPCSecurityGroup  
  LambdaEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      ServiceName: !Sub 'com.amazonaws.${AWS::Region}.lambda'
      VpcId: !Ref CustomVPC
      SubnetIds: 
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroupIds:
        - !Ref VPCSecurityGroup
  EbsEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      ServiceName: !Sub 'com.amazonaws.${AWS::Region}.ebs'
      VpcId: !Ref CustomVPC
      SubnetIds: 
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      SecurityGroupIds:
        - !Ref VPCSecurityGroup

CustomSNSTopic:- An Amazon SNS topic is a logical access point that acts as a communication channel. A topic lets you group multiple endpoints (such as AWS Lambda, Amazon SQS, HTTP/S, or an email address).
DisplayName:- The display name to use for an Amazon SNS topic with SMS subscriptions.
Subscription:- The Amazon SNS subscriptions (endpoints) for this topic.
Endpoint:-The endpoint which needs to be subscribed as part of this sns topic.
Protocol:- Type of protocol using which we want to subscribe.
TopicName:- The name of the topic you want to create. Topic names must include only uppercase and lowercase ASCII letters, numbers, underscores, and hyphens, and must be between 1 and 256 characters long.

  CustomSNSTopic: 
    Type: AWS::SNS::Topic
    Properties: 
      DisplayName: "CustomSNS"
      Subscription: 
        - Endpoint: "<your email id>"
          Protocol: "email"
      TopicName: "CustomSNS"

UnusedElasticIpDetectionLambdaRole:- AWS Identity and Access Management (IAM) roles provide a way to access AWS by relying on temporary security credentials. Each role has a set of permissions for making AWS service requests, and a role is not associated with a specific user or group.
RoleName: Specify rolename you want to create.
AssumeRolePolicyDocument: The purpose of the AssumeRolePolicyDocument is to contain the trust relationship policy that grants an entity permission to assume the role.
ManagedPolicyArns: Pick policy which is managed by AWS Directly.
Policies: A policy is an object in AWS that, when associated with an entity or resource, defines their permissions. AWS evaluates these policies when a principal, such as a user, makes a request. Permissions in the policies determine whether the request is allowed or denied. Most policies are stored in AWS as JSON documents.

  UnusedElasticIpDetectionLambdaRole: 
    Type: "AWS::IAM::Role"
    Properties: 
      RoleName: "UnusedElasticIpDetectionLambdaRole"
      AssumeRolePolicyDocument: 
        Version: "2012-10-17"
        Statement: 
          - 
            Effect: "Allow"
            Principal: 
              Service: 
                - "lambda.amazonaws.com"
            Action: 
              - "sts:AssumeRole"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
      Policies:
        - PolicyName: "UnusedElasticIpDetectionLambdaPolicy"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - "sns:ListSubscriptionsByTopic"
                  - "sns:GetTopicAttributes"
                  - "sns:ListSMSSandboxPhoneNumbers"
                  - "logs:*"
                  - "sns:ListTopics"
                  - "sns:ListSubscriptions"
                  - "ec2:DescribeVolumeAttribute"
                  - "sns:ListOriginationNumbers"
                  - "sns:ListEndpointsByPlatformApplication"
                  - "sns:Publish"
                  - "ec2:DescribeAddressesAttribute"
                  - "ec2:DescribeAddresses"
                  - "ec2:DescribeInstanceAttribute"
                  - "sns:ListPlatformApplications"
                Resource: '*'

UnusedElasticIpDetectionLambda:- Creating Security group and enabling ingress with http and ssh port.
Runtime:- This property is used to mention security group name.
MemorySize:- This property is used to mention security group description and its mandatory property for this resource.
Timeout:- This property is used to add ingress rules for [udp/tcp] ports enabled secured access to your resources.
Handler:- One of the most important property used in all resources. Always make sure to attach tags for all your resources.
Code:- One of the most important property used in all resources. Always make sure to attach tags for all your resources.
Code:- One of the most important property used in all resources. Always make sure to attach tags for all your resources.

  UnusedElasticIpDetectionLambda:
    Type: AWS::Lambda::Function
    Properties:
      MemorySize: 320
      Timeout: 300
      Runtime: python3.8
      Role: !GetAtt UnusedElasticIpDetectionLambdaRole.Arn
      Handler: index.handler
      Code:
        ZipFile: |
          import boto3
          import os
          sns_client = boto3.client('sns')
          ec2 = boto3.client('ec2')
          response = ec2.describe_addresses()
          unused_eips = []         
          def handler(event, context):
           # First we will typecast response and save in variable to check if there is any existing elastic IP
           check = list(response['Addresses'])
           if not check:
               print("Elastic IP does not exist | Exiting program.....")
               exit()         
            # If address is available we will check if it is associated with instance or not
            for address in response['Addresses']:
                if 'InstanceId' in address:
                    print('Elastic IP {} is associated with instance {}'.format(address['PublicIp'], address['InstanceId']))
                else:
                    print('Elastic IP {} is unused and can be released'.format(address['PublicIp']))
                    unused_eips.append("Unused elastic ip: {}".format(address['PublicIp']))
            #email
            sns_topic_arn = os.environ['Topic_ARN']
            sns_client.publish(
                TopicArn= sns_topic_arn,
                Subject='Warning - Unused Elastic Ip's List',
                Message=str(unused_eips)
            )
            return "success"
      Description: Invoke a function during stack creation.
      Environment: 
        Variables: 
          Topic_ARN: !Ref CustomSNSTopic
      TracingConfig:
        Mode: Active
      VpcConfig:
        SecurityGroupIds:
          - !Ref LambdaSecurityGroup
        SubnetIds:
          - !Ref PublicSubnet1
          - !Ref PublicSubnet2

PermissionForEventsToInvokeLambda:- The AWS::Lambda::Permission resource grants an AWS service or another account permission to use a function. You can apply the policy at the function level, or specify a qualifier to restrict access to a single version or alias. If you use a qualifier, the invoker must use the full Amazon Resource Name (ARN) of that version or alias to invoke the function.
FunctionName: The name of the Lambda function, version, or alias.
Action: The action that the principal can use on the function.
Principal: The AWS service or account that invokes the function. If you specify a service, use SourceArn or SourceAccount to limit who can invoke the function through that service.
SourceArn: For AWS services, the ARN of the AWS resource that invokes the function.

  PermissionForEventsToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: !GetAtt UnusedEbsDetectionLambda.Arn
      Action: "lambda:InvokeFunction"
      Principal: "events.amazonaws.com"
      SourceArn: !GetAtt UnusedEbsDetectionScheduledRule.Arn

UnusedElasticIpDetectionScheduledRule:- Creates or updates the specified rule. Rules are enabled by default, or based on value of the state.
Description: The description of the rule.
ScheduleExpression:The scheduling expression. For example, "cron(0 20 ? )", "rate(5 minutes)". .
State: The state of the rule.
*Targets
: Adds the specified targets to the specified rule, or updates the targets if they are already associated with the rule.Targets are the resources that are invoked when a rule is triggered.

  UnusedElasticIpDetectionScheduledRule: 
    Type: AWS::Events::Rule
    Properties: 
      Description: "ScheduledRule"
      ScheduleExpression: "cron(0 11 ? * 7 *)"
      State: "ENABLED"
      Targets: 
        - Arn: !GetAtt UnusedElasticIpDetectionLambda.Arn
          Id: "UnusedElasticIpDetectionLambda"

🔳 Outputs

Its always a best practice to print output for your resources.
outputVPCSecurityGroup:- A reference to the created VPC Security Group.
outputLambdaSecurityGroup:- A reference to the created Lambda Security Group. ✦ outputCustomSNSTopic: A reference to the created SNS Topic.
outputUnusedElasticIpDetectionLambdaRole: A reference to the created Lambda Role.
outputUnusedElasticIpDetectionLambda: A reference to the created Lambda.
outputPermissionForEventsToInvokeLambda: A reference to the created Permission to invoke lambda.
outputUnusedElasticIpDetectionScheduledRule: A reference to the created Event Rule.

Outputs: 
  outputVPCSecurityGroup: 
    Description: A reference to the created VPC Security Group
    Value: !Ref VPCSecurityGroup
  outputLambdaSecurityGroup: 
    Description:  A reference to the created Lambda Security Group
    Value: !Ref LambdaSecurityGroup
  outputCustomSNSTopic: 
    Description: A reference to the created SNS Topic
    Value: !Ref CustomSNSTopic
  outputUnusedElasticIpDetectionLambdaRole: 
    Description: A reference to the created Lambda Role
    Value: !Ref UnusedElasticIpDetectionLambdaRole
  outputUnusedElasticIpDetectionLambda: 
    Description: A reference to the created Lambda
    Value: !Ref UnusedElasticIpDetectionLambda
  outputPermissionForEventsToInvokeLambda: 
    Description: A reference to the created Permission to invoke lambda
    Value: !Ref PermissionForEventsToInvokeLambda
  outputUnusedElasticIpDetectionScheduledRule: 
    Description: A reference to the created Event Rule
    Value: !Ref UnusedElasticIpDetectionScheduledRule

🔊 To view entire github code click here

Unused Elastic IP (1).png

1️⃣ Lets validate our template 👨‍💻

aws cloudformation validate-template --template-body file://<file path>

2️⃣ After successful template verification lets create stack using our template 👨‍💻

aws cloudformation create-stack --stack-name elasticip --template-body file://<file path>

3️⃣ Check if the stack we created via template is completed successfully 👨‍💻

aws cloudformation list-stack-resources --stack-name elasticip

4️⃣ Describe stack and its resources to view its properties 👨‍💻

aws cloudformation describe-stacks --stack-name elasticip
aws cloudformation describe-stack-resources --stack-name elasticip

5️⃣ Check events for stack formation 👨‍💻

aws cloudformation describe-stack-events --stack-name elasticip

3.png

4.png

❗️❗️Important AWS Documentation To Be Viewed❗️❗️

⛔️ Lambda Invoke Permission
⛔️ Events Rule
⛔️ VPC Endpoint
⛔️ SNS Topic
⛔️ Lambda Function
⛔️ Parameters
⛔️ Outputs

🥁🥁 Conclusion 🥁🥁

In this blog we have deployed below components
✦ Security Groups For Lambda And VPC.
✦ VPC Endpoints.
✦ SNS Topic.
✦ IAM Role For Lambda.
✦ Lambda Function to fetch non associated Elastic Ip's and notify using SNS.
I have used AWS CLI command to deploy these template and trust me AWS CLI is the realtime hero and I would suggest you to get acquainted towards it. Going forward I will be releasing further parts to this CloudFormation journey

📢Stay tuned for my next blog.....

🎊So, did you find my content helpful? If you did or like my other content, feel free to buy me a coffee. Thanks. 🎊

💫Cloudformation Series Sequence💫

🔰 Deploy VPC With Internet Gateway & Associate I
🔰 Public, Private Subnet & Route Table Creation & Association II
🔰 Private Subnet,Nat Gateway, Elastic Ip, Private Route Table & Associate III
🔰 NACL, Inbound & Outbound Routes, Security Group & Associate With Subnet IV
🔰 EC2 With Security Group & User Data & Mapping V
🔰 Target Group, Elastic Load Balancer & ELB Listener VI
🔰 Build Web Application Layer With AWS CloudFormation VII

⌛️Realtime Usecases Cloudformation Templates⏳

💨 Schedule Automatic Detection Of Unused AWS EBS Volumes & Notify
💨 Schedule Automatic Detection Of Non Associated AWS Elastic IP's In AWS Account On Weekly Basis And Notify
💨 Schedule Automatic Deregistration Of AWS AMI On Weekly Basis And Notify

👨🏻‍💻Cloudformation Github Repository👨🏻‍💻

Did you find this article valuable?

Support Dheeraj Choudhary by becoming a sponsor. Any amount is appreciated!

See recent sponsors Learn more about Hashnode Sponsors
 
Share this