DevSecops Project jenkins pipeline deploying in EKS and moniotoring with Grafana
find out source code in github repo
Step1: Build Infrastructure
terraform init
terraform apply
Step2: Setup Prometheus and Grafana On kubernetes Cluster
go to directory of k8s
first install Custom Resouce Defination for Prometheus Operator
kubectl create -f /prometheus-operator-crd
then install prometheus Operator
After installing whole Prometheus setup lets setup jenkins server
Step1: Install Jenkins, Docker and Trivy
Connect to your console, and enter these commands to Install Jenkins
#!/bin/bash
sudo apt update -y
#sudo apt upgrade -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
/usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins
Unlock Jenkins and install suggested Plugins
Jenkins Getting Started Screen.
Install Docker
sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker $USER #my case is ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock
After the docker installation, we create a sonarqube container (Remember to add 9000 ports in the security group).
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community
Now our sonarqube is up and running
Install Trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
Step3: Create a TMDB API key
Next, we will create a TMDB API key
Open a new tab in the Browser and search for TMDB
once you create an account you will see this page.
Let's create an API key, By clicking on your profile and clicking settings.
Now click on API from the left side panel.click on Developer
Accept this aggremment.
:
Fill form
you will get your api key copy and save it.
Step4: Email Integration With Jenkins and Plugin Setup
Install Email Extension Plugin in Jenkins
Go to your Gmail and click on your profile
Then click on Manage Your Google Account --> click on the security tab on the left side panel you will get this page(provide mail password).
Click on other and provide your name and click on Generate and copy the password
Once the plugin is installed in Jenkins, click on manage Jenkins --> configure system there under the E-mail Notification section configure the details as shown in the below image
Click on Apply and save.
Click on Manage Jenkins--> credentials and add your mail username and generated password
post {
always {
emailext attachLog: true,
subject: "'${currentBuild.result}'",
body: "Project: ${env.JOB_NAME}<br/>" +
"Build Number: ${env.BUILD_NUMBER}<br/>" +
"URL: ${env.BUILD_URL}<br/>",
to: 'postbox.aj99@gmail.com', #change Your mail
attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
}
}
Next, we will log in to Jenkins and start to configure our Pipeline in Jenkins.
Step5: Install Plugin like JDK, Sonarqube Scanner , Nodejs , Owasp Dependency
Goto Manage Jenkins →Plugins → Available Plugins →
Install below plugins
1 → Eclipse Temurin Installer (Install without restart)
2 → SonarQube Scanner (Install without restart)
3 → NodeJs Plugin (Install Without restart)
Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save
Step6: Configure Sonar Server in Manage Jenkins.
Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, so <Public IP>:9000. Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token
click on update Token
Create a token with a name and generate
copy Token
Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this
Now, go to Dashboard → Manage Jenkins → System and Add like the below image.
Click on Apply and Save
The Configure System option is used in Jenkins to configure different server
Global Tool Configuration is used to configure different tools that we install using Plugins
We will install a sonar scanner in the tools.
In the Sonarqube Dashboard add a quality gate also
Administration--> Configuration-->Webhooks
Click on Create
Let's go to our Pipeline and add the script in our Pipeline Script.
pipeline{
agent any
tools{
jdk 'jdk17'
nodejs 'node16'
}
environment {
SCANNER_HOME=tool 'sonar-scanner'
}
stages {
stage('clean workspace'){
steps{
cleanWs()
}
}
stage('Checkout from Git'){
steps{
git branch: 'main', url: 'https://github.com/harish343/netflix-clone-react-typescript.git'
}
}
stage("Sonarqube Analysis "){
steps{
withSonarQubeEnv('sonar-server') {
sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
-Dsonar.projectKey=Netflix '''
}
}
}
stage("quality gate"){
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
}
}
}
stage('Install Dependencies') {
steps {
sh "npm install"
}
}
}
post {
always {
emailext attachLog: true,
subject: "'${currentBuild.result}'",
body: "Project: ${env.JOB_NAME}<br/>" +
"Build Number: ${env.BUILD_NUMBER}<br/>" +
"URL: ${env.BUILD_URL}<br/>",
to: 'harishnarnaware75@gmail.com',
attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
}
}
}
Click on Build now, you will see the stage view like this
Step7: Install OWASP Dependency Check Plugins
GotoDashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restart.
First, we configured the Plugin and next, we had to configure the Tool
Goto Dashboard → Manage Jenkins → Tools →
Click on Apply and Save here.
Now go configure → Pipeline and add this stage to your pipeline and build.
stage('OWASP FS SCAN') {
steps {
dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
stage('TRIVY FS SCAN') {
steps {
sh "trivy fs . > trivyfs.txt"
}
}
The stage view would look like this,
You will see that in status, a graph will also be generated and Vulnerabilities.
Step8: Docker Image Build and Push
We need to install the Docker tool in our system, Goto Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step
Now, goto Dashboard → Manage Jenkins → Tools →
Add DockerHub Username and Password under Global Credentials
Add this stage to Pipeline Script
stage("Docker Build & Push"){
steps{
script{
withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){
sh "docker build --build-arg TMDB_V3_API_KEY=Aj7ay86fe14eca3e76869b92 -t netflix ."
sh "docker tag netflix sevenajay/netflix:latest "
sh "docker push sevenajay/netflix:latest "
}
}
}
}
stage("TRIVY"){
steps{
sh "trivy image sevenajay/netflix:latest > trivyimage.txt"
}
}
Step9: Deploy on EKS
FIrst configure your aws cli on Jenkins Machine.
Install Kubernetes Plugin, Once it's installed successfully
goto manage Jenkins --> manage credentials --> Click on Jenkins global --> add credentials
stage('Deploy to kubernets'){
steps{
script{
dir('Kubernetes') {
withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
sh 'kubectl apply -f deployment.yml'
sh 'kubectl apply -f service.yml'
}
}
}
}
}