DevOps Practice

• 1. Project overview
• 2. What I learned
• 3. Things to do

• Project Overview

Launched a Drupal website that hosting on AWS with EC2 instance http://52.11.193.136/  .  Created a tool for testing some basic functionalities of the Drupal site, which includes 1.User login/out, 2. Adding new content to website 3. All links on the website are functional( 200 OK response or similar). 4. User posting comments. 5.The search bar is responsible. 6.Database connection. 7.Requesting new user account.  ( I tried Amazon CodeCommit for version control, but I prefer GitHub. So I maintain this code on my GitHub repo).

Connected Drupal site with an RDS MySql database instance and an S3 bucket to make sure we are able to upload and download relatively large amounts of data online.

Create a Dockerfile and then a docker image based on the first Drupal site on AWS, and upload it into EC2 Container Service(ECS) which allows us to load-balance and future deployment easily. Utilized EC2 container service to create a launch configuration and then launched 2 similar EC2 instances on AWS ( 2 other similar Drupal site http://52.11.112.135/ and http://35.167.78.83/ ).Besides, also connect them with a new RDS MySql instance and S3 Cloud Storage bucket to ensure a working persistent filesystem.

Create an auto scaling group based on the ECS container we created before. Set some scaling policies, which allows AWS CloudWatch Service monitor our EC2 instance and auto-scale our group when there is an alarm triggered.  

Last but not least, we also want our website is Highly Available to users all the time. From the architectural design view, we could build a fault tolerance system where allows a backup deployment exist, but that may cost us too much.Instead we could get help from lots of AWS related services. I created another EC2 instance in different AZ( Available Zone ) and formed an application load balancer for the Drupal site at http://sitebalancer-2044199213.us-west-2.elb.amazonaws.com/ This load balancer will enable balancing our traffic in between this two server and when one of them have a down time for some reason, another one could still serving.

Actually, the auto scaling group we created before will also ensure our website is highly available when we have an overwhelming request to our server. Besides, choosing a relatively big size of the database and keep backing up important user data is also crucial for us to achieve high availability.

• 2. What I learned

For me, many things need to learn from scratch in order to complete this challenge. To name a few, how to launch a Drupal site on an EC2 instance? how to connect it with DB and S3? How do we perform testings on a website? What is Docker? What is a REST API ? … etc. However, I really enjoy the whole learning process, which gives me an opportunity to gain some handy skills very quickly and broaden my horizon of interests.

Specifically, For Drupal, I learned how to launch and manage a Drupal website through AWS Linux instance. How to connect Drupal site with its dependent services. Major pros and cons for using a Drupal framework.

For AWS services, I  learned how to launch and manage multiple EC2 instances. How to connect an EC2 instance with other related AWS service ( RDS, S3 ). How to configure and deploy our workloads using EC2 Container Service. How to create an auto-scaling group and enable AWS CloudWatch service. Then, set related scaling policies to achieve scaling up under load. Finally, How to create and manage an load balancer with EC2 instances that locate in different Available Zone.

For website testing, I learned how to use and customize Drupal SimpleTest Framework. ( Few PHP test files that I created with SimpleTest were attached below). How to use Selenium WebDriver to achieve web browser automation test. How to create automation test framework. How to use logging infrastructure when testing (log4j). How to connect Database with Selenium WebDriver. My tool for this challenge was created with Selenium WebDriver.  I will also attach a sample log.out file ( log created by my tool )  and a README  file that describe how to compile and run my tool along with it.  

Besides those three major aspects, Fortunately, I also learn how to work with Dockerfile and Docker image. How to make our website Highly Available. What is and why we want to our server work as a REST API.

• 3.Things to do

If there is more time that I can spend with this challenge or the further steps that could  improve this project I will do the following.

Add more test cases in my tool that it could test a Drupal website in a more comprehensive way.Also, because RESTful Web Services is included in Drupal 8 core, I also want to use my tool to test it as a REST API.     

Keep enhancing the availability of our website host by trying to integrate our system with AWS Elastic Load Balancing service and set up a database replication in our MySQL database. Besides, I would like to try some third party tools that in AWS marketplace to add an extra layer of our fault tolerance system.  

Finally, get more understanding and appreciation of tools, such as Puppet and Docker. Know more about how we, as a software developer, could better use them to configure workloads and managing a production cluster.

Thanks.

Ye Jiang

2/11/2017