Complete Guide to End-to-End DevOps Implementation on a MERN Stack Application

TLDR: This blog post provides a comprehensive guide to implementing an end-to-end DevOps pipeline for a MERN stack application, covering infrastructure automation with Terraform, CI/CD with Jenkins and Argo CD, and monitoring with Prometheus and Grafana.

In this blog post, we will explore how to implement an end-to-end DevOps pipeline for a MERN stack application. This guide will cover the entire process, from infrastructure automation to continuous integration and delivery, and finally to monitoring. By the end of this post, you will have a clear understanding of how to set up a complete DevOps workflow.

The MERN stack is a popular technology stack for building web applications. It consists of:

• MongoDB: A NoSQL database for storing application data.

• Express.js: A web application framework for Node.js, used for building the backend.

• React.js: A JavaScript library for building user interfaces, used for the frontend.

• Node.js: A JavaScript runtime for executing server-side code.

This stack allows developers to use JavaScript throughout the entire application, making it easier to manage and develop.

GitHub Repo for Application Code and Jenkins Files

GitHub Repo for Terraform files to create EKS Cluster

Complete Documentation for the Project

Overview of the Architecture

The architecture for our DevOps implementation is designed to automate the entire process. Here’s a brief overview of the components involved:

• Terraform: Used for infrastructure automation.

• Jenkins: Used for continuous integration (CI).

• Argo CD: Used for continuous delivery (CD) to the Kubernetes cluster.

• Prometheus and Grafana: Used for monitoring the application.

properties([
    parameters([
        string(
            defaultValue: 'dev',
            name: 'Environment'
        ),
        choice(
            choices: ['plan', 'apply', 'destroy'], 
            name: 'Terraform_Action'
        )])
])
pipeline {
    agent any
    stages {
        stage('Preparing') {
            steps {
                sh 'echo Preparing'
            }
        }
        stage('Git Pulling') {
            steps {
                git branch: 'master', url: 'https://github.com/AmanPathak-DevOps/EKS-Terraform-GitHub-Actions.git'
            }
        }
        stage('Init') {
            steps {
                withAWS(credentials: 'aws-creds', region: 'us-east-1') {
                sh 'terraform -chdir=eks/ init'
                }
            }
        }
        stage('Validate') {
            steps {
                withAWS(credentials: 'aws-creds', region: 'us-east-1') {
                sh 'terraform -chdir=eks/ validate'
                }
            }
        }
        stage('Action') {
            steps {
                withAWS(credentials: 'aws-creds', region: 'us-east-1') {
                    script {    
                        if (params.Terraform_Action == 'plan') {
                            sh "terraform -chdir=eks/ plan -var-file=${params.Environment}.tfvars"
                        }   else if (params.Terraform_Action == 'apply') {
                            sh "terraform -chdir=eks/ apply -var-file=${params.Environment}.tfvars -auto-approve"
                        }   else if (params.Terraform_Action == 'destroy') {
                            sh "terraform -chdir=eks/ destroy -var-file=${params.Environment}.tfvars -auto-approve"
                        } else {
                            error "Invalid value for Terraform_Action: ${params.Terraform_Action}"
                        }
                    }
                }
            }
        }
    }
}

jenkins-code

1. parameters

   1. string → defaultValue : dev , name

   2. choice → choices: [‘plan‘,’apply’,’destroy’] , name

2. pipelines

   1. agent

   2. stages

      1. Preparing

      2. Git Pulling

      3. Init

      4. Validate

      5. Action

         1. plan

         2. apply

         3. destroy

step 2

step 3

Step 1: Automating Infrastructure with Terraform

The first step in our DevOps pipeline is to automate the infrastructure setup using Terraform. This involves creating an AWS environment with the following components:

• An EC2 instance to host Jenkins.

• A Virtual Private Cloud (VPC) to isolate our resources.

• An Elastic Kubernetes Service (EKS) cluster to run our application.

Setting Up the EC2 Instance

1. Launch an EC2 instance using Terraform.

2. Install necessary tools such as Jenkins, Docker, and Terraform on the instance.

3. Configure security groups to allow traffic on required ports (e.g., Jenkins on port 8080).

Creating the VPC and EKS Cluster

1. Use Terraform scripts to create a VPC with subnets.

2. Deploy an EKS cluster within the VPC, ensuring that it is private and secure.

3. Set up a jump server to manage access to the EKS cluster.

Step 2: Implementing Continuous Integration with Jenkins

Once the infrastructure is set up, we move on to implementing CI using Jenkins. This involves several stages:

Stages of the CI Pipeline

1. Code Checkout: Pull the application code from a GitHub repository.

2. Code Quality Analysis: Use SonarQube to analyze the code for quality and security issues.

3. Dependency Checks: Run OWASP dependency checks to identify vulnerabilities.

4. File Scanning: Scan for unnecessary files in the codebase.

5. Docker Image Creation: Build Docker images for both the frontend and backend applications.

6. Pushing Docker Images To Private ECR

7. Image Scanning: Use Trivy to scan the Docker images for vulnerabilities in ECR.

8. Update Kubernetes Manifests: Update the deployment manifests with the new image tags.

Setting Up Jenkins

• Install Jenkins and necessary plugins for Docker, SonarQube, and AWS integration.

• Configure Jenkins credentials for accessing AWS and GitHub.

• Create a Jenkins pipeline that automates the above stages.

Step 3: Continuous Delivery with Argo CD

After the CI pipeline is successfully executed, we will set up continuous delivery using Argo CD. This involves:

1. Deploying Applications: Use Argo CD to deploy the frontend, backend, and database applications to the EKS cluster.

2. Managing Application State: Argo CD will ensure that the deployed applications match the desired state defined in the GitHub repository.

3. Using Ingress for External Access: Configure Ingress resources to expose the applications to the internet.

Step 4: Monitoring with Prometheus and Grafana

The final step in our DevOps pipeline is to set up monitoring for our applications using Prometheus and Grafana:

1. Install Prometheus: Deploy Prometheus on the EKS cluster to collect metrics from the applications.

2. Install Grafana: Deploy Grafana to visualize the metrics collected by Prometheus.

3. Create Dashboards: Set up Grafana dashboards to monitor application performance and health.

Conclusion

In this blog post, we have covered the complete process of implementing an end-to-end DevOps pipeline for a MERN stack application. From automating infrastructure with Terraform to setting up CI/CD with Jenkins and Argo CD, and finally monitoring with Prometheus and Grafana, you now have a comprehensive understanding of how to manage a modern web application using DevOps practices.

For further details, please refer to the documentation and GitHub repository linked in the description. Happy coding!