DevOps CI/CD Pipelines: Automated Testing, Building, and Deployment with GitHub Actions
13 min readby imnb
Build production-grade CI/CD pipelines with GitHub Actions. Automate testing, security scanning, Docker builds, and deployments to AWS, Azure, and Kubernetes with practical examples.
Modern DevOps requires automated pipelines that test, build, and deploy code reliably. This guide covers building CI/CD pipelines with GitHub Actions, from basic workflows to complex multi-environment deployments.
Basic CI Pipeline: Test and Build
yaml
# .github/workflows/ci.yml
name: CI Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
NODE_VERSION: 18
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- name: Install dependencies
run: npm ci # Faster than npm install
- name: Run linter
run: npm run lint
- name: Run tests
run: npm test -- --coverage
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
file: ./coverage/coverage-final.json
fail_ci_if_error: true
build:
runs-on: ubuntu-latest
needs: test # Only build if tests pass
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
- run: npm ci
- run: npm run build
- name: Upload build artifacts
uses: actions/upload-artifact@v3
with:
name: build
path: dist/
retention-days: 7Docker Build and Push
yaml
# .github/workflows/docker.yml
name: Build and Push Docker Image
on:
push:
branches: [main]
tags:
- 'v*'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Log in to Container Registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,prefix={{branch}}-
- name: Build and push
uses: docker/build-push-action@v4
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
platforms: linux/amd64,linux/arm64
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload Trivy results
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'Multi-Environment Deployment
yaml
# .github/workflows/deploy.yml
name: Deploy to Environments
on:
push:
branches: [main, staging]
workflow_dispatch: # Manual trigger
inputs:
environment:
description: 'Environment to deploy'
required: true
type: choice
options:
- staging
- production
jobs:
deploy-staging:
if: github.ref == 'refs/heads/staging' || github.event.inputs.environment == 'staging'
runs-on: ubuntu-latest
environment:
name: staging
url: https://staging.example.com
steps:
- uses: actions/checkout@v3
- name: Deploy to Staging
run: |
echo "Deploying to staging..."
# Deploy commands here
deploy-production:
if: github.ref == 'refs/heads/main' || github.event.inputs.environment == 'production'
runs-on: ubuntu-latest
environment:
name: production
url: https://example.com
needs: [test, security-scan] # Gates before production
steps:
- uses: actions/checkout@v3
- name: Deploy to Production
run: |
echo "Deploying to production..."
# Deploy commands here
- name: Create release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: v${{ github.run_number }}
release_name: Release v${{ github.run_number }}
draft: false
prerelease: false
- name: Notify Slack
uses: 8398a7/action-slack@v3
if: always()
with:
status: ${{ job.status }}
text: 'Production deployment ${{ job.status }}'
webhook_url: ${{ secrets.SLACK_WEBHOOK }}AWS Deployment Pipeline
yaml
# .github/workflows/aws-deploy.yml
name: Deploy to AWS
on:
push:
branches: [main]
jobs:
deploy-to-ecs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Build and push image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/my-app:$IMAGE_TAG .
docker push $ECR_REGISTRY/my-app:$IMAGE_TAG
- name: Update ECS service
run: |
aws ecs update-service \
--cluster my-cluster \
--service my-service \
--force-new-deployment
- name: Wait for deployment
run: |
aws ecs wait services-stable \
--cluster my-cluster \
--services my-service
deploy-lambda:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
- name: Install dependencies
run: npm ci --production
- name: Package Lambda
run: zip -r function.zip .
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Deploy to Lambda
run: |
aws lambda update-function-code \
--function-name my-function \
--zip-file fileb://function.zip
- name: Publish new version
run: |
aws lambda publish-version \
--function-name my-function
- name: Update alias
run: |
VERSION=$(aws lambda list-versions-by-function \
--function-name my-function \
--query 'Versions[-1].Version' --output text)
aws lambda update-alias \
--function-name my-function \
--name production \
--function-version $VERSIONSecurity Scanning
yaml
# .github/workflows/security.yml
name: Security Scans
on:
push:
branches: [main, develop]
schedule:
- cron: '0 0 * * 0' # Weekly
jobs:
dependency-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run npm audit
run: npm audit --audit-level=high
- name: Check for known vulnerabilities
uses: snyk/actions/node@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
code-scanning:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: javascript
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
secrets-scanning:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Gitleaks scan
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}Best Practices
- Use environment-specific secrets (staging vs production)
- Implement approval gates for production deployments
- Cache dependencies to speed up workflows (setup-node cache option)
- Use matrix strategy to test multiple Node/Python versions
- Set workflow-level timeouts to prevent hanging jobs
- Use concurrency controls to cancel outdated runs
- Store artifacts for debugging failed builds
- Monitor workflow costs (Actions minutes) and optimize
- Use self-hosted runners for private code or faster builds
- Implement blue-green or canary deployments for zero-downtime
- Use deployment protection rules for critical environments
- Tag images with git sha for traceability
- Run security scans on every PR before merge
- Use composite actions to reuse workflow logic
- Implement automatic rollback on deployment failure