DevOps CI/CD Pipelines: Automated Testing, Building, and Deployment with GitHub Actions

Modern DevOps requires automated pipelines that test, build, and deploy code reliably. This guide covers building CI/CD pipelines with GitHub Actions, from basic workflows to complex multi-environment deployments.

Basic CI Pipeline: Test and Build

1# .github/workflows/ci.yml
2name: CI Pipeline
3
4on:
5  push:
6    branches: [main, develop]
7  pull_request:
8    branches: [main]
9
10env:
11  NODE_VERSION: 18
12
13jobs:
14  test:
15    runs-on: ubuntu-latest
16    
17    steps:
18      - name: Checkout code
19        uses: actions/checkout@v3
20      
21      - name: Setup Node.js
22        uses: actions/setup-node@v3
23        with:
24          node-version: ${{ env.NODE_VERSION }}
25          cache: 'npm'
26      
27      - name: Install dependencies
28        run: npm ci  # Faster than npm install
29      
30      - name: Run linter
31        run: npm run lint
32      
33      - name: Run tests
34        run: npm test -- --coverage
35      
36      - name: Upload coverage
37        uses: codecov/codecov-action@v3
38        with:
39          file: ./coverage/coverage-final.json
40          fail_ci_if_error: true
41  
42  build:
43    runs-on: ubuntu-latest
44    needs: test  # Only build if tests pass
45    
46    steps:
47      - uses: actions/checkout@v3
48      - uses: actions/setup-node@v3
49        with:
50          node-version: ${{ env.NODE_VERSION }}
51      
52      - run: npm ci
53      - run: npm run build
54      
55      - name: Upload build artifacts
56        uses: actions/upload-artifact@v3
57        with:
58          name: build
59          path: dist/
60          retention-days: 7

Docker Build and Push

1# .github/workflows/docker.yml
2name: Build and Push Docker Image
3
4on:
5  push:
6    branches: [main]
7    tags:
8      - 'v*'
9
10env:
11  REGISTRY: ghcr.io
12  IMAGE_NAME: ${{ github.repository }}
13
14jobs:
15  build-and-push:
16    runs-on: ubuntu-latest
17    permissions:
18      contents: read
19      packages: write
20    
21    steps:
22      - name: Checkout
23        uses: actions/checkout@v3
24      
25      - name: Set up Docker Buildx
26        uses: docker/setup-buildx-action@v2
27      
28      - name: Log in to Container Registry
29        uses: docker/login-action@v2
30        with:
31          registry: ${{ env.REGISTRY }}
32          username: ${{ github.actor }}
33          password: ${{ secrets.GITHUB_TOKEN }}
34      
35      - name: Extract metadata
36        id: meta
37        uses: docker/metadata-action@v4
38        with:
39          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
40          tags: |
41            type=ref,event=branch
42            type=semver,pattern={{version}}
43            type=semver,pattern={{major}}.{{minor}}
44            type=sha,prefix={{branch}}-
45      
46      - name: Build and push
47        uses: docker/build-push-action@v4
48        with:
49          context: .
50          push: true
51          tags: ${{ steps.meta.outputs.tags }}
52          labels: ${{ steps.meta.outputs.labels }}
53          cache-from: type=gha
54          cache-to: type=gha,mode=max
55          platforms: linux/amd64,linux/arm64
56      
57      - name: Run Trivy vulnerability scanner
58        uses: aquasecurity/trivy-action@master
59        with:
60          image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
61          format: 'sarif'
62          output: 'trivy-results.sarif'
63      
64      - name: Upload Trivy results
65        uses: github/codeql-action/upload-sarif@v2
66        with:
67          sarif_file: 'trivy-results.sarif'

Multi-Environment Deployment

1# .github/workflows/deploy.yml
2name: Deploy to Environments
3
4on:
5  push:
6    branches: [main, staging]
7  workflow_dispatch:  # Manual trigger
8    inputs:
9      environment:
10        description: 'Environment to deploy'
11        required: true
12        type: choice
13        options:
14          - staging
15          - production
16
17jobs:
18  deploy-staging:
19    if: github.ref == 'refs/heads/staging' || github.event.inputs.environment == 'staging'
20    runs-on: ubuntu-latest
21    environment:
22      name: staging
23      url: https://staging.example.com
24    
25    steps:
26      - uses: actions/checkout@v3
27      
28      - name: Deploy to Staging
29        run: |
30          echo "Deploying to staging..."
31          # Deploy commands here
32  
33  deploy-production:
34    if: github.ref == 'refs/heads/main' || github.event.inputs.environment == 'production'
35    runs-on: ubuntu-latest
36    environment:
37      name: production
38      url: https://example.com
39    needs: [test, security-scan]  # Gates before production
40    
41    steps:
42      - uses: actions/checkout@v3
43      
44      - name: Deploy to Production
45        run: |
46          echo "Deploying to production..."
47          # Deploy commands here
48      
49      - name: Create release
50        uses: actions/create-release@v1
51        env:
52          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
53        with:
54          tag_name: v${{ github.run_number }}
55          release_name: Release v${{ github.run_number }}
56          draft: false
57          prerelease: false
58      
59      - name: Notify Slack
60        uses: 8398a7/action-slack@v3
61        if: always()
62        with:
63          status: ${{ job.status }}
64          text: 'Production deployment ${{ job.status }}'
65          webhook_url: ${{ secrets.SLACK_WEBHOOK }}

AWS Deployment Pipeline

1# .github/workflows/aws-deploy.yml
2name: Deploy to AWS
3
4on:
5  push:
6    branches: [main]
7
8jobs:
9  deploy-to-ecs:
10    runs-on: ubuntu-latest
11    
12    steps:
13      - uses: actions/checkout@v3
14      
15      - name: Configure AWS credentials
16        uses: aws-actions/configure-aws-credentials@v2
17        with:
18          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
19          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
20          aws-region: us-east-1
21      
22      - name: Login to Amazon ECR
23        id: login-ecr
24        uses: aws-actions/amazon-ecr-login@v1
25      
26      - name: Build and push image
27        env:
28          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
29          IMAGE_TAG: ${{ github.sha }}
30        run: |
31          docker build -t $ECR_REGISTRY/my-app:$IMAGE_TAG .
32          docker push $ECR_REGISTRY/my-app:$IMAGE_TAG
33      
34      - name: Update ECS service
35        run: |
36          aws ecs update-service \
37            --cluster my-cluster \
38            --service my-service \
39            --force-new-deployment
40      
41      - name: Wait for deployment
42        run: |
43          aws ecs wait services-stable \
44            --cluster my-cluster \
45            --services my-service
46      
47  deploy-lambda:
48    runs-on: ubuntu-latest
49    
50    steps:
51      - uses: actions/checkout@v3
52      - uses: actions/setup-node@v3
53      
54      - name: Install dependencies
55        run: npm ci --production
56      
57      - name: Package Lambda
58        run: zip -r function.zip .
59      
60      - name: Configure AWS credentials
61        uses: aws-actions/configure-aws-credentials@v2
62        with:
63          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
64          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
65          aws-region: us-east-1
66      
67      - name: Deploy to Lambda
68        run: |
69          aws lambda update-function-code \
70            --function-name my-function \
71            --zip-file fileb://function.zip
72      
73      - name: Publish new version
74        run: |
75          aws lambda publish-version \
76            --function-name my-function
77      
78      - name: Update alias
79        run: |
80          VERSION=$(aws lambda list-versions-by-function \
81            --function-name my-function \
82            --query 'Versions[-1].Version' --output text)
83          
84          aws lambda update-alias \
85            --function-name my-function \
86            --name production \
87            --function-version $VERSION

Security Scanning

1# .github/workflows/security.yml
2name: Security Scans
3
4on:
5  push:
6    branches: [main, develop]
7  schedule:
8    - cron: '0 0 * * 0'  # Weekly
9
10jobs:
11  dependency-check:
12    runs-on: ubuntu-latest
13    
14    steps:
15      - uses: actions/checkout@v3
16      
17      - name: Run npm audit
18        run: npm audit --audit-level=high
19      
20      - name: Check for known vulnerabilities
21        uses: snyk/actions/node@master
22        env:
23          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
24  
25  code-scanning:
26    runs-on: ubuntu-latest
27    
28    steps:
29      - uses: actions/checkout@v3
30      
31      - name: Initialize CodeQL
32        uses: github/codeql-action/init@v2
33        with:
34          languages: javascript
35      
36      - name: Perform CodeQL Analysis
37        uses: github/codeql-action/analyze@v2
38  
39  secrets-scanning:
40    runs-on: ubuntu-latest
41    
42    steps:
43      - uses: actions/checkout@v3
44        with:
45          fetch-depth: 0
46      
47      - name: Gitleaks scan
48        uses: gitleaks/gitleaks-action@v2
49        env:
50          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Best Practices

Use environment-specific secrets (staging vs production)
Implement approval gates for production deployments
Cache dependencies to speed up workflows (setup-node cache option)
Use matrix strategy to test multiple Node/Python versions
Set workflow-level timeouts to prevent hanging jobs
Use concurrency controls to cancel outdated runs
Store artifacts for debugging failed builds
Monitor workflow costs (Actions minutes) and optimize
Use self-hosted runners for private code or faster builds
Implement blue-green or canary deployments for zero-downtime
Use deployment protection rules for critical environments
Tag images with git sha for traceability
Run security scans on every PR before merge
Use composite actions to reuse workflow logic
Implement automatic rollback on deployment failure