HomeНаука и техникаRelated VideosMore From: Chandra Shettigar

Create CloudFormation Stack (EC2 + RDS) & Deploy Docker App - Episode #7

422 ratings | 26540 views
Learn how to create an application stack using AWS CloudFormation. Stack will consist of EC2 instance and RDS for MySQL. Stack is defined using YAML template that contains 4 resources to be created on AWS, 1. EC2 Instance to run the application on 2. EC2 Security Group that defines the possible inbound ports on the EC2 instance 3. RDS Instance for MySQL database 4. RDS Security Group that defines the possible sources the db can be reached from https://devteds.com/episodes/7-create-aws-cloudformation-stack-for-ec2-rds-and-deploy-docker-app Following are the steps to walk through, 1. Create a KeyPair on AWS account to be used for SSH access to EC2 2. Generate API keys and configure the AWS CLI on local machine 3. Create a simple stack using CloudFormation and this stack will consists of an EC2 instance and necessary security group for EC2 instance 4. Update the stack to install and configure docker on the EC2 instance 5. Update the stack to add RDS instance and necessary security group to the stack 6. Create Docker compose file to define the application to be deployed on EC2 instance with it’s database on RDS instance 7. Deploy the application on the EC2 instance (docker host) remotely from local ------------------------------------------------------------------------ Source code: https://github.com/devteds/e7-cloudformation-docker Docker Image: https://hub.docker.com/r/devteds/rails-api-example ------------------------------------------------------------------------ Related episodes: Episode #9 - Docker on Amazon ECS with Fargate using CloudFormation https://youtu.be/Gr2yTSsVSqg Episode #6 - Deploy Dockerized Web API application on Amazon EC2 with RDS https://youtu.be/bU9o2fVreRU Episode #3 - APIs on Rails: Create a rails API-only application https://youtu.be/pTgaCgfH6_U ------------------------------------------------------------------------ Commands & README: https://github.com/devteds/e7-cloudformation-docker/blob/master/README.md ------------------------------------------------------------------------ Resources: AWS CloudFormation: https://aws.amazon.com/cloudformation AWS EC2: https://aws.amazon.com/ec2 AWS RDS: https://aws.amazon.com/rds Docker Image: https://hub.docker.com/r/devteds/rails-api-example Configure Docker Engine: https://docs.docker.com/engine/admin/ Docker & TLS: https://docs.docker.com/engine/security/https/ ------------------------------------------------------------------------ Author: Chandra Shettigar | Software Engineer with over decade of experience writing softwares. https://devteds.com, https://www.linkedin.com/in/shettigarc, https://twitter.com/shettigar ------------------------------------------------------------------------
Html code for embedding videos on your blog
Text Comments (72)
Rihan Pereira (1 month ago)
This is magnificient. I absorbed the know-how in under 25 min sitting. Thank you for your work, chandra!
Pandian Palanimuthu (1 month ago)
Exquisite! Thanks for sharing.
Aditya Gyawali (1 month ago)
HELP!!!! I have a NODE application with POSTGRE database, I am also using knex.js for querying data. How can I use database migration with the way you used for a rails application. I have followed your video and everything is the same except the application i am using is NODE.
Pluasok (1 month ago)
Right speed for a typical human brain to consume without buffering
GetOutOfHereStalkan (1 month ago)
This is shockingly well made.
Pankaj Sarswat (1 month ago)
excellent video man, thanks for sharing your knowledge
Chandra Shettigar (1 month ago)
Thank you Pankaj
Christian-Peter Heimbach (3 months ago)
I really enjoyed that. Given how many painful tutorials are out there. You rock. Thanks.
Chandra Shettigar (3 months ago)
Thank you Christian 🙂
Sagar Srivastava (4 months ago)
great presentation style, keep up the good work
Chandra Shettigar (3 months ago)
Thanks for the kind words, Sagar.
hi (6 months ago)
really super one!!
Chandra Shettigar (4 months ago)
Thank you.
Siddhesh Kerkar (6 months ago)
Awesome!!! Precise and Clear..Thanks
Chandra Shettigar (6 months ago)
Thank you Siddhesh.
Santosh Lohar (7 months ago)
Excellent way of explanation !! thanks for sharing this.
Chandra Shettigar (6 months ago)
Thank you Santosh.
Kashish Sharma (8 months ago)
hi your video very nice and helpful, could you please help me to install Java 8 and maven and tomcat on the ec2 instance using cloud formation. thanks in advance please create a video or share any documents.
Chandra Shettigar (8 months ago)
Thank you Kashish. I am glad you found the videos useful. I don't do much Java but I have that in my list of videos to make. I can't be sure of timeline as when I can publish one on Java. In the meantime if I come across any tutorials or blogs on Java 8 + Maven + Cloudformation, I will send them to you.
prashant shetage (9 months ago)
Hi. Your videos are very helpful. I am referring those for my daily tasks. Could you please make a video on CloudFormation for EMR with "Bootstrap Scripts(Postgres/MySql DB installation) & Step Execution(Spark Job)" Thanks
Chandra Shettigar (9 months ago)
Thank you Prashant. I appreciate your feedback and request. When time permits I will look into making a tutorial on EMR but I can't really promise now. If I can't make the video, I will try to post some code examples on https://github.com/devteds .
sarfaraz shaikh (10 months ago)
Excellent, Please create more videos on Cloudformation using cli.
Chandra Shettigar (10 months ago)
Thank you Sarfaraz. Please check Episode #9 (https://youtu.be/Gr2yTSsVSqg) which is also a CloudFormation CLI but for ECS stack.
vijay b (1 year ago)
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "AWS CloudFormation Sample Template EC2InstanceWithSecurityGroup", "Resources": { "Ec2Instance": { "Type": "AWS::EC2::Instance", "Properties": { "InstanceType": "t2.micro", "SecurityGroups": [{ "Ref": "InstanceSecurityGroup" }], "KeyName": "JENKINS", "ImageId": "ami-922914f7", "UserData": { "Fn::Base64": { "Fn::Join": [ "\n", [ "#!/bin/bash", "sudo -i", "yum update", "yum install wget -y", " wget https://pkg.jenkins.io/redhat-stable/jenkins-1.642.2-1.1.noarch.rpm -y", "rpm -ivh jenkins-1.642.2-1.1.noarch.rpm", " service jenkins start" ] ] } } } }, "InstanceSecurityGroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "Enable SSH access via port 22", "SecurityGroupIngress": [{ "IpProtocol": "tcp", "FromPort": "22", "ToPort": "22", "CidrIp": "0.0.0.0/0" }] } } } } Launching EC2,but unable to install Jenkins..plz help
vijay b (1 year ago)
Hi sir,,, am unable to install Jenkins via cloud formation... Plz help with script for installing Jenkins via cloud formation
Master Programming AZ (1 year ago)
hi man, thanks for sharing. I have a question that what should I do/ follow/ learn if I want to be an DevOps? --thx a lot
Chandra Shettigar (1 year ago)
Thanks for asking my suggestion but I don't think I can be of much help providing guidance for a devops career path. But I will send you some information and links that might help.
Vikas Rathore (1 year ago)
Great tutorial. Thanks Chanda. Could you please guide me how I can create a docker image and push it to ECR using cloudformation. Thanks in advance
Chandra Shettigar (1 year ago)
Thanks for the feedback Vikas. I am not exactly sure I understood your question. If you're trying to automate the process of creating docker image on every code commit and that you want that image be pushed over to the registry, that would be to create a simple or part of CI (Continuous Integration) pipeline. If that's what you're looking to build using CloudFormation and entirely on AWS, here is a high level of what goes into CloudFormation stack definition, 1. Create code repository resource (AWS::CodeCommit::Repository); You would use this as your source repo. 2. Create container image registry resource (AWS::ECR::Repository); This is where you would want the docker images pushed. 3. Create code build project resource (AWS::CodeBuild::Project), the source for this will be CodeCommit repo. Environment of CodeBuild project is where you would define the instructions to build docker image and push to registry. You will probably need to define buildspec.yml in the root of your code repo with necessary instructions to build & push. You may configure code build project to trigger on every code commit. I haven't used CodeBuild but if I were to create a CI on AWS, this is how I would go about. Now you can use Github code repository instead of CodeCommit in which case the Source definition of CodeBuild project will require some additional inputs. I hope this helps.
Joshua Trampier (1 year ago)
Thanks for tutorial. How to auto-populate RDS hostname in docker-compose yml file, instead of waiting and pasting?
Chandra Shettigar (1 year ago)
This is not a recommend approach for a real production stack but if you are just trying to automate the process of what is being done in this tutorial, one quick option would be to use CloudFormation Output variable. You can define CloudFormation output, value of which will be "!GetAtt DatabaseInstance.Endpoint.Address". Once the stack is created, you could write a small shell script to describe stack (using AWS CLI - aws cloudformation describe-stack ..), grep the output value and assign it to an env variable. In the compose file app.yml, read the database endpoint from that environment variable. I don't think you can avoid waiting but have the script wait until the RDS instance is ready before starting docker app. As I mentioned in this short tutorial, you could use CloudFormation itself to automate your stack (infra & app). Eg: To define and automate tracking of completion state of resources and then deploy the application. ECS is one good option where you don't deal with docker compose but define all the container services in CloudFormation template.
Venkat K (1 year ago)
Excellent Chandra.
Chandra Shettigar (1 year ago)
Thanks Venkat.
helina freesgi (1 year ago)
How do I access the terminal that you are talking about?
Chandra Shettigar (1 year ago)
Sorry for the delayed reply. I hope you were able to resolve the issue by now. If not, please refer my response to ddubs123's (one of the viewers) question.
helina freesgi (1 year ago)
I keep receiving the below error at cloudformation and the stack gets rolled_back everytime: The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request. Client Request Token:Console-CreateStack-fe29ceb9-82d9-4f4a-9f06-2a3402fd8999 The following resource(s) failed to create: [AppNode]. . Rollback requested by user.
Chandra Shettigar (1 year ago)
If you are using mac, you may use the "Terminal" app. The one I use on mac is called "iTerm". If you use windows 10, I think there are some good options such as "Bash on Ubuntu on Windows 10" and OpenSSH client. In the older windows, I don't know if there many options besides Putty or Cygwin.
Ram M (1 year ago)
use Putty. then SSH <instance or ip>
maheeza sharma (1 year ago)
Awesome , good job chandra
Chandra Shettigar (1 year ago)
Thank you Maheeza Sharma
RICHARD JOY (1 year ago)
Very informative and well explained. Thank you.
Chandra Shettigar (1 year ago)
Thanks Richard Joy. I'm glad it was helpful
ddubs123 (1 year ago)
Using your config, I got an error while the task was creating the EC2 instance. "The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request."
Chandra Shettigar (1 year ago)
Likely there is no default VPC. If there isn't one already, you might want to create a default vpc and try again. FYI - To create default on aws management console - under services, go to VPCs and from 'Actions' dropdown you should be able to create a new default vpc. Make sure to delete the failed stack before you try running 'aws cloudformation create ... " command. Hope this helps.
Velu Natarajan (1 year ago)
Thanks for quick overview on CloudFormation....this helped me lot !!
Chandra Shettigar (1 year ago)
Thanks for the feedback Velu.
dheeraj tripathi (1 year ago)
Just 20 minutes of this session and I now find it way more easier, thanks :)
Chandra Shettigar (1 year ago)
Thanks Dheeraj. I'm glad it was helpful.
Ashutosh Utpat (1 year ago)
Fantastic!!! Very useful information.
Chandra Shettigar (1 year ago)
Thanks for the feedback Ashutosh.
Thimma Reddy (1 year ago)
How can add extra root volume to the instance when it is creating
Thimma Reddy (1 year ago)
Chandra Shettigar I am creating an Beanstalk Env by using cloud formation, when it is launching instance it has to add 20 GB root volume to it
Chandra Shettigar (1 year ago)
Thimma Reddy, Did you mean EBS block device mapping or create some directories to mount to the container? If it's about creating directories to mount to containers, you can add those commands in the UserData. If you're looking to add EBS kinda stuff, you might want to refer the CloudFormation EC2 reference documentation & examples. Following links might help - http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html#cfn-ec2-instance-blockdevicemappings http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-ec2.html
shivaji gandi (1 year ago)
I had one question in docker.How the developers work culture would be using containerization concept like check in and checkouts and how the developer environment is to be created on demand to test their own code etc
shivaji gandi (1 year ago)
Thanks
Chandra Shettigar (1 year ago)
In general containerized development environment helps the developers in several ways, 1. Consistent runtime, libraries, database versions - that means developers don't have to worry about installing, upgrading software versions and focus on application development. No tooling blockers and focus on getting the things done. 2. Consistent OS - this is a big plus. Developers can use any OS of their choice in the local environment. The basic OS & libraries that is needed for application is packaged in the container. This helps when you have a mix of developers that have preference for their primary OS. With the application dockerized, setting up environments on demand or the first time one a developer machine is all quick an easy. You probably need to write some scripts to seed data to the environment you spin up. If the application is not run as containers in production you can still containerize in development and CI environments to use the exact same versions of OS, libraries and softwares. That still helps. Sometimes it is hard to introduce containerization or docker in a dev team. For some it is some new tool to learn which takes back seat. But once introduced the team will notice improvement in productivity. I hope that answers your question.
shivaji gandi (1 year ago)
I like the way you demonstrate this video.Good job
Chandra Shettigar (1 year ago)
Thanks Shivaji. I hope this was useful too.
Orest Ivasiv (1 year ago)
well done.
Chandra Shettigar (1 year ago)
Thank you Orest Ivasiv.
Suresh Kumar (1 year ago)
Awesome. Too good explanation.
Chandra Shettigar (1 year ago)
Thank you Suresh.
Ravi C (1 year ago)
I like the pace of your tutorial. This would easily be 2 hours long if some of those other guys do it.
Chandra Shettigar (1 year ago)
Thank you Ravi.
rajdeeprandhawa1 (1 year ago)
Very good presentation Chandra - very simple and elegant
Chandra Shettigar (1 year ago)
Thank you rajdeeprandhawa1
Bennette Stephen John (2 years ago)
For Production, I would chose CloudFormation + ECS/Elastic Beanstalk + RDS. With ECS/Elastic Beanstalk, you can directly deploy your container, without having to worry about the docker host, set/change environment variables during deploy, easy to scale up/down
Chandra Shettigar (1 year ago)
Yes I agree - for production or even some non-production environments, I would pick CloudFormation + ECS + RDS. The purpose of this video was to demonstrate CloudFormation with some use-case for which I chose Docker installation. I was expecting a comment and feedback like this :) I am thinking to make a few videos around "CloudFormation + ECS + AutoScale" which I think might be a series of short videos or a course. Thanks!
Bennette Stephen John (2 years ago)
I like the way you edit your videos, which makes it crisp, clear and concise. If I may suggest, 3:47-4:00 you should mask your AWS secrets as anyone can use it to hack your AWS account (hope you deleted the IAM user used in this video :) )
Chandra Shettigar (1 year ago)
Thanks for the feedback and suggestions @Bennette Stephen John. I thought of masking the AWS secrets but I chose to clean up all the resources and api keys etc after finishing up the video. In some cases I think masking will still be necessary. Thanks!
Satish Kumar (2 years ago)
awesome
Chandra Shettigar (2 years ago)
Thank you Satish.

Would you like to comment?

Join YouTube for a free account, or sign in if you are already a member.