User Tools
Writing /app/www/public/data/meta/development/applications/ansible/aws-ansible.meta failed
development:applications:ansible:aws-ansible
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| development:applications:ansible:aws-ansible [2018/06/22 11:37] – adowling | development:applications:ansible:aws-ansible [2021/06/25 10:09] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | --- // | ||
| + | |||
| + | ====== AWS Automation with Ansible ====== | ||
| + | |||
| + | Bitbucket Repo: https:// | ||
| + | |||
| + | Other Available SDK's for AWS: https:// | ||
| + | |||
| + | ===== Overview ===== | ||
| + | * This project covers the vast majority of playbooks needed in order to automate the server environment configuration for the IDMS on AWS. | ||
| + | * This project uses ansible with boto3 to automate resource generation and configuration for many aspects of our AWS environment setup aside from vpn and vpc. | ||
| + | * The project also handles the cleanup and deletion of these resources when the environment is not in use. These can be of benefit for qa environments that are not always needed. | ||
| + | * The original request for this project was so AmericanTower' | ||
| + | |||
| + | ===== Benefits / Goals of this Project (Present and Future)===== | ||
| + | - Provides a repository and means of change tracking to git for our AWS environments, | ||
| + | - Follows AWS's Infrastructure as code management approach. | ||
| + | - Forces user to think about what they are setting up and running by reading the playbooks and identifying what vars are needed for their resource setup. | ||
| + | - Decreases likelihood of accidental resource deletion or mistaken additions which manual UI usage can cause. | ||
| + | |||
| + | ===== Ansible AWS Documentation: | ||
| + | |||
| + | https:// | ||
| + | ===== Setup ===== | ||
| + | The Following things need to be installed on your local machine prior to using this project :!: | ||
| + | |||
| + | - Clone the ansible-aws repository: git clone git@bitbucket.org: | ||
| + | - Install python: sudo yum install python (Should be installed by default on mac osx) | ||
| + | - Install ansible: sudo yum install ansible 2.5+ | ||
| + | - Install pip: sudo yum install python-pip | ||
| + | - sudo python -m pip install boto3 | ||
| + | - Before running any playbooks ensure a group_vars directory is setup with the correct variables for your aws environment (e.g. qaatc, nvqa, newenv etc.). Take a look at the other group_vars files to determine what vars are needed to run the playbooks (these include things like the vpc-id, naming conventions for resources, tags for resources, subnet specifications etc.) | ||
| + | |||
| + | ===== API Credentials ===== | ||
| + | There are a number of ways to run ansible scripts against aws. The method chosen for this project was to package the credentials within the environment variables. As we have multiple AWS accounts currently, we also have multiple environments set up in the project (qaatc, nvqa). These each have a vault file containing the access key and secret key for that account. These are fully encrypted and require a password to run all scripts. Passwords can be found in PasswordSafe under **AWS ANSIBLE VAULT ENVIRONMENTS**. | ||
| + | |||
| + | ==== Order of Playbooks: ==== | ||
| + | **Due to dependencies of certain things being created at certain stages, the following is the ordering in which to run the creation and deletion playbooks associated with this project.** | ||
| + | |||
| + | === Creation: === | ||
| + | - createRDSInstance.yml | ||
| + | - QA Transform scripts | ||
| + | - createEC2Instance.yml | ||
| + | - createTargetGroup.yml | ||
| + | - createALBAndAssignRules.yml | ||
| + | |||
| + | === Deletion: === | ||
| + | - deleteRDSInstance.yml | ||
| + | - deleteEC2Instance.yml | ||
| + | - deleteALB.yml | ||
| + | - deleteTargetGroup.yml | ||
| + | |||
| + | ===== RDS ===== | ||
| + | https:// | ||
| + | |||
| + | ==== CREATE ==== | ||
| + | //Restore an RDS Instance from a snapshot and adds a route 53 entry for that rds instance to resolve the host name to its rds endpoint.// | ||
| + | |||
| + | Command line Parameters: | ||
| + | * rds_snapshot(the snapshot name) | ||
| + | * add_route53=' | ||
| + | < | ||
| + | ansible-playbook createRDSInstance.yml -e " | ||
| + | </ | ||
| + | |||
| + | ==== DELETE ==== | ||
| + | //Delete an RDS instance from a specified name.// | ||
| + | |||
| + | Command line Parameters: | ||
| + | * rds_instance_name=rds: | ||
| + | < | ||
| + | ansible-playbook deleteRDSInstance.yml -e " | ||
| + | </ | ||
| + | |||
| + | ===== EC2 ===== | ||
| + | - https:// | ||
| + | - https:// | ||
| + | - https:// | ||
| + | |||
| + | ==== CREATE ==== | ||
| + | //Create an EC2 Instances inside a vpc assigned to a subnet group.// | ||
| + | |||
| + | Command line Parameters: | ||
| + | * volume_size: | ||
| + | * instance_type: | ||
| + | * region: the region to deploy in (n.virginia(us-east-1), | ||
| + | * subnet_id: the subnet to create and assign the ip to server in | ||
| + | * ami: the image to create the server from. | ||
| + | * name: the name to assign the ec2 instance. | ||
| + | * keypair: the .pem file name | ||
| + | * private_ip: the subnet private ip to assign the instance. | ||
| + | * add_elastic_ip: | ||
| + | |||
| + | < | ||
| + | Command LB1: | ||
| + | |||
| + | Command APPS1: | ||
| + | |||
| + | Command APPS2: | ||
| + | </ | ||
| + | |||
| + | ==== DELETE ==== | ||
| + | //Delete an EC2 Instances by specifying its id.// | ||
| + | |||
| + | Command line Parameters: | ||
| + | * specified_instance_id: | ||
| + | < | ||
| + | ansible-playbook deleteEC2Instance.yml -e " | ||
| + | </ | ||
| + | |||
| + | ==== REBOOT ==== | ||
| + | //Reboot an EC2 Instances by specifying its id.// | ||
| + | |||
| + | Command line Parameters: | ||
| + | * specified_instance_id: | ||
| + | < | ||
| + | ansible-playbook rebootEC2.yml -e " | ||
| + | </ | ||
| + | |||
| + | ===== Application Load Balancer ===== | ||
| + | - https:// | ||
| + | - https:// | ||
| + | |||
| + | ==== CREATE ==== | ||
| + | //Create Application Load Balancer, assign the SSL Certificate from its ARN id (the cert should be uploaded to AWS prior to this) and Assign Target Group forwarding Rules to the ALB.// | ||
| + | < | ||
| + | ansible-playbook -e " | ||
| + | </ | ||
| + | |||
| + | ==== DELETE ==== | ||
| + | //Delete the Application Load Balancer(name is specified in the group_vars for the appropriate environment). This deletion step must be run before deleting the target groups//. | ||
| + | < | ||
| + | ansible-playbook -e " | ||
| + | </ | ||
| + | |||
| + | ===== Target Groups ===== | ||
| + | https:// | ||
| + | |||
| + | ==== CREATE ==== | ||
| + | //Create Application Load Balancer Target Groups for Applications and EC2 Handlers to forward traffic for different apps between the correct handlers and distribute load.// | ||
| + | < | ||
| + | ansible-playbook -e " | ||
| + | </ | ||
| + | |||
| + | ==== DELETE ==== | ||
| + | //Deletes application load balancer target groups.// | ||
| + | |||
| + | < | ||
| + | ansible-playbook -e " | ||
| + | </ | ||
| + | |||
| + | ===== EC2 Volume Snapshots ===== | ||
| + | Repository url: https:// | ||
| + | This project covers all the playbooks needed in order to automate the volume snapshot process for our AWS EC2 servers. | ||
| + | This provides us with a means of failover for server storage if a recovery scenario was needed. | ||
| + | |||
| + | Details: | ||
| + | - Runs on NVQALB1 & ATCLB1 currently | ||
| + | - Permissions to run this ansible script are handled through IAM role assignments on the instance so no API keys are stored for security reasons. This also allevaited the problem of not being able to use a password with the cronjob. | ||
| + | |||
| + | ## Scheduling | ||
| + | The process is be scheduled to run via cron using the create_ec2_snapshots.sh file (ensure this file is executable using chmod +x) | ||
| + | The correct environment playbook should be specified in the bash script ansible command (i.e. nvqa, atcprod). The tag options for create and deleting snapshots must be set to true in order for the process to work. | ||
| + | |||
| + | < | ||
| + | ansible-playbook playbooks/ | ||
| + | |||
| + | 01 00 * * * / | ||
| + | </ | ||
| + | |||
| + | ===== Testing Overview ===== | ||
| + | |||
| + | https:// | ||
| + | |||
| + | ===== Future Plans ===== | ||
| + | - **SSL** - Load SSL certs directly to aws, server keystores and to tomcat containers via ansible. | ||
| + | - **VPC** - Create a VPC (virtual private cloud) with assigned security groups and subnet definitions via ansible. | ||
| + | - **VPN** - Create a VPN for aws via ansible and associated all static routes and rules with a vpc. | ||
| + | - **Security Group Rules** - Automate the creation and maintenance of security group rules so that we have a central repository from which changes can be applied and tracked. | ||
| + | - **IAM Roles** - Automate the creation and management of IAM roles and policies. | ||
| + | - **CloudWatch** - Automate the creation and management of Cloudwatch Monitoring Rules for various scenarios. | ||
Page Tools
