motovaultpro/docs/CICD-DEPLOY.md

MotoVaultPro GitLab CI/CD Deployment Guide

Complete guide for deploying MotoVaultPro using GitLab CI/CD with shell executor runners.

Table of Contents

1. Prerequisites
2. GitLab Runner Setup
3. CI/CD Variables Configuration
4. Secrets Architecture
5. Pipeline Overview
6. Deployment Process
7. Rollback Procedure
8. Troubleshooting

---

Prerequisites

Server Requirements

• Linux server with Docker Engine installed
• Docker Compose v2 (plugin version)
• GitLab Runner installed and registered
• Git installed
• curl installed (for health checks)

GitLab Requirements

• GitLab 18.6+ (tested with 18.6.2)
• Project with CI/CD enabled
• Protected main branch
• Maintainer access for CI/CD variable configuration

---

GitLab Runner Setup

1. Verify Runner Registration

sudo gitlab-runner verify

Expected output should show your runner as active with shell executor.

2. Verify Docker Permissions

The gitlab-runner user must have Docker access:

# Add gitlab-runner to docker group (if not already done)
sudo usermod -aG docker gitlab-runner

# Verify access
sudo -u gitlab-runner docker info
sudo -u gitlab-runner docker compose version

3. Verify Deployment Directory

Ensure the deployment directory exists and is accessible:

# Create deployment directory
sudo mkdir -p /opt/motovaultpro
sudo chown gitlab-runner:gitlab-runner /opt/motovaultpro

# Clone repository (first time only)
sudo -u gitlab-runner git clone <repository-url> /opt/motovaultpro

---

CI/CD Variables Configuration

Navigate to Settings > CI/CD > Variables in your GitLab project.

Secrets (File Type Variables)

These variables use GitLab's File type, which writes the value to a temporary file and provides the path as the environment variable. This replicates the Kubernetes secrets pattern used by the application.

Variable Name	Type	Protected	Masked	Description
POSTGRES_PASSWORD	File	Yes	Yes	PostgreSQL database password
AUTH0_CLIENT_SECRET	File	Yes	Yes	Auth0 client secret for backend
GOOGLE_MAPS_API_KEY	File	Yes	Yes	Google Maps API key
GOOGLE_MAPS_MAP_ID	File	Yes	No	Google Maps Map ID
CF_DNS_API_TOKEN	File	Yes	Yes	Cloudflare API token for Let's Encrypt DNS challenge
RESEND_API_KEY	File	Yes	Yes	Resend API key for email notifications

Configuration Variables

Variable Name	Type	Protected	Masked	Value
VITE_AUTH0_DOMAIN	Variable	No	No	motovaultpro.us.auth0.com
VITE_AUTH0_CLIENT_ID	Variable	No	No	Your Auth0 client ID
VITE_AUTH0_AUDIENCE	Variable	No	No	https://api.motovaultpro.com

Note: DEPLOY_PATH is automatically set in .gitlab-ci.yml using GIT_CLONE_PATH for a stable path.

Creating Cloudflare API Token

The CF_DNS_API_TOKEN is required for automatic SSL certificate generation via Let's Encrypt DNS-01 challenge.

1. Go to Cloudflare Dashboard
2. Click Create Token
3. Use template: Edit zone DNS
4. Configure permissions:
   • Permissions: Zone > DNS > Edit
   • Zone Resources: Include > Specific zone > motovaultpro.com
5. Click Continue to summary then Create Token
6. Copy the token value immediately (it won't be shown again)
7. Add as CF_DNS_API_TOKEN File variable in GitLab

Setting Up a File Type Variable

1. Go to Settings > CI/CD > Variables
2. Click Add variable
3. Enter the variable key (e.g., POSTGRES_PASSWORD)
4. Enter the secret value in the Value field
5. Set Type to File
6. Enable Protect variable (recommended)
7. Enable Mask variable (for sensitive data)
8. Click Add variable

---

Secrets Architecture

MotoVaultPro uses a Kubernetes-style secrets pattern where secrets are mounted as files at /run/secrets/ inside containers.

How It Works

1. GitLab stores secrets as File type CI/CD variables
2. During pipeline execution, GitLab writes each secret to a temporary file
3. The inject-secrets.sh script copies these files to secrets/app/ directory
4. Docker Compose mounts these files to /run/secrets/ in containers
5. Application code reads secrets from the filesystem (not environment variables)

Secret Files

secrets/app/
  postgres-password.txt       -> /run/secrets/postgres-password
  auth0-client-secret.txt     -> /run/secrets/auth0-client-secret
  google-maps-api-key.txt     -> /run/secrets/google-maps-api-key
  google-maps-map-id.txt      -> /run/secrets/google-maps-map-id
  cloudflare-dns-token.txt    -> /run/secrets/cloudflare-dns-token
  resend-api-key.txt          -> /run/secrets/resend-api-key

Security Benefits

• Secrets never appear as environment variables (not visible in env or printenv)
• File permissions (600) restrict access
• Masked variables prevent accidental log exposure
• Protected variables only available on protected branches

---

Pipeline Overview

The CI/CD pipeline consists of four stages:

Stage 1: Validate

Verifies deployment prerequisites:

• Docker is accessible
• Docker Compose is available
• Deployment directory exists

Stage 2: Build

Builds Docker images:

• Pulls latest code from repository
• Builds all service images with --no-cache

Stage 3: Deploy

Deploys the application:

1. Injects secrets from GitLab variables
2. Stops existing services gracefully
3. Pulls base images
4. Starts database services (PostgreSQL, Redis)
5. Runs database migrations
6. Starts all services

Stage 4: Verify

Validates deployment health:

• Checks all containers are running
• Tests backend health endpoint
• Reports deployment status

Pipeline Diagram

[Validate] -> [Build] -> [Deploy] -> [Verify]
     |            |          |           |
   Check       Build      Inject      Health
  prereqs     images     secrets      checks
                            |
                         Migrate
                            |
                         Start
                        services

---

Deployment Process

Automatic Deployment

Deployments are triggered automatically when:

• Code is pushed to the main branch
• A merge request is merged into main

Manual Deployment

To trigger a manual deployment:

1. Go to CI/CD > Pipelines
2. Click Run pipeline
3. Select the main branch
4. Click Run pipeline

Deployment Steps (What Happens)

1. Secrets Injection

   • inject-secrets.sh copies GitLab File variables to secrets/app/
   • Permissions are set to 600 for security

2. Service Shutdown

   • Existing containers are stopped gracefully (30s timeout)
   • Volumes are preserved

3. Database Startup

   • PostgreSQL and Redis start first
   • 15-second wait for database readiness

4. Migrations

   • Backend container runs database migrations
   • Ensures schema is up-to-date

5. Full Service Startup

   • All services start via docker compose up -d
   • Traefik routes traffic automatically

6. Health Verification

   • Container status checks
   • Backend health endpoint validation

---

Rollback Procedure

Automatic Rollback

If the verify stage fails, the pipeline will report failure but services remain running. Manual intervention is required.

Manual Rollback

Use the rollback script:

# SSH to server
ssh user@server

# Run rollback to previous commit
cd /opt/motovaultpro
./scripts/rollback.sh HEAD~1

# Or rollback to specific tag/commit
./scripts/rollback.sh v1.0.0

Rollback Script Details

The script performs:

1. Stops all current services
2. Checks out the specified version
3. Rebuilds images
4. Starts services

Emergency Recovery

If rollback fails:

cd /opt/motovaultpro

# Stop everything
docker compose down

# Check git history
git log --oneline -10

# Checkout known working version
git checkout <commit-hash>

# Rebuild and start
docker compose build
docker compose up -d

# Verify
docker compose ps

---

Troubleshooting

Pipeline Fails at Validate Stage

Symptom: DEPLOY_PATH not found

Solution:

# Create directory on runner server
sudo mkdir -p /opt/motovaultpro
sudo chown gitlab-runner:gitlab-runner /opt/motovaultpro

Pipeline Fails at Build Stage

Symptom: Docker build errors

Solutions:

1. Check Dockerfile syntax
2. Verify network connectivity for npm/package downloads
3. Check disk space: df -h
4. Clear Docker cache: docker system prune -a

Pipeline Fails at Deploy Stage

Symptom: Secrets injection fails

Solutions:

1. Verify CI/CD variables are configured correctly
2. Check variable types are set to File for secrets
3. Ensure variables are not restricted to specific environments

Symptom: Migration fails

Solutions:

1. Check database connectivity
2. Verify PostgreSQL is healthy: docker logs mvp-postgres
3. Run migrations manually:

   docker compose exec mvp-backend npm run migrate
   

Pipeline Fails at Verify Stage

Symptom: Container not running

Solutions:

1. Check container logs: docker logs <container-name>
2. Verify secrets are correctly mounted
3. Check for port conflicts

Symptom: Health check fails

Solutions:

1. Wait longer (service might be starting)
2. Check backend logs: docker logs mvp-backend
3. Verify database connection

Services Start But Application Doesn't Work

Check secrets are mounted:

docker compose exec mvp-backend ls -la /run/secrets/

Check configuration:

docker compose exec mvp-backend cat /app/config/production.yml

Check network connectivity:

docker network ls
docker network inspect motovaultpro_backend

Viewing Logs

# All services
docker compose logs -f

# Specific service
docker compose logs -f mvp-backend

# Last 100 lines
docker compose logs --tail 100 mvp-backend

---

Maintenance

Updating Secrets

1. Update the CI/CD variable in GitLab
2. Trigger a new pipeline (push or manual)
3. The new secrets will be injected during deployment

Database Backups

Backups should be configured separately. Recommended approach:

# Manual backup
docker compose exec mvp-postgres pg_dump -U postgres motovaultpro > backup.sql

# Automated backup (add to cron)
0 2 * * * cd /opt/motovaultpro && docker compose exec -T mvp-postgres pg_dump -U postgres motovaultpro > /backups/mvp-$(date +\%Y\%m\%d).sql

Monitoring

Consider adding:

• Prometheus metrics (Traefik already configured)
• Health check alerts
• Log aggregation

---

Quick Reference

Common Commands

# View pipeline status
# GitLab UI: CI/CD > Pipelines

# SSH to server
ssh user@your-server

# Navigate to project
cd /opt/motovaultpro

# View running containers
docker compose ps

# View logs
docker compose logs -f

# Restart a service
docker compose restart mvp-backend

# Run migrations manually
docker compose exec mvp-backend npm run migrate

# Access database
docker compose exec mvp-postgres psql -U postgres motovaultpro

# Health check
curl http://localhost:3001/health

Important Paths

Path	Description
$CI_BUILDS_DIR/motovaultpro	Application root (stable clone path)
$CI_BUILDS_DIR/motovaultpro/secrets/app/	Secrets directory
$CI_BUILDS_DIR/motovaultpro/data/documents/	Document storage
$CI_BUILDS_DIR/motovaultpro/config/	Configuration files

Note: CI_BUILDS_DIR is typically /opt/gitlab-runner/builds for shell executors.

Container Names

Container	Purpose
mvp-traefik	Reverse proxy, TLS termination
mvp-frontend	React SPA
mvp-backend	Node.js API
mvp-postgres	PostgreSQL database
mvp-redis	Redis cache