fix: CI/CD blue-green deployment path bug causing stale production content

Root cause: switch-traffic.sh was modifying Traefik config in the CI checkout
directory ($GITHUB_WORKSPACE) instead of the deployment directory ($DEPLOY_PATH).
Additionally, the sed patterns didn't work with multi-line YAML structure.

Changes:
- Add DEPLOY_PATH environment variable support to all CI scripts
- Add --force-recreate flag to ensure containers are recreated with new images
- Fix weight update to use awk for reliable multi-line YAML editing
- Add scripts/ directory to rsync so SREs can run scripts from /opt/motovaultpro
- Add image verification step to confirm containers use expected images
- Add weight verification to confirm Traefik routing was updated
- Add routing validation step to verify traffic switch succeeded

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

.gitea/workflows/production.yaml

@@ -95,9 +95,10 @@ jobs:
           sparse-checkout-cone-mode: false
           fetch-depth: 1
 
-      - name: Sync config and compose files to deploy path
+      - name: Sync config, scripts, and compose files to deploy path
         run: |
           rsync -av --delete "$GITHUB_WORKSPACE/config/" "$DEPLOY_PATH/config/"
+          rsync -av --delete "$GITHUB_WORKSPACE/scripts/" "$DEPLOY_PATH/scripts/"
           cp "$GITHUB_WORKSPACE/docker-compose.yml" "$DEPLOY_PATH/"
           cp "$GITHUB_WORKSPACE/docker-compose.blue-green.yml" "$DEPLOY_PATH/"
 

README.md

@@ -26,3 +26,8 @@ make migrate  # run DB migrations
 ## URLs and Hosts
 - Frontend: `https://motovaultpro.com`
 - Backend health: `https://motovaultpro.com/api/health`
+
+## Operational Commands
+- View active environment on production: `sudo cat /opt/motovaultpro/config/deployment/state.json`
+- Switch traffic between environments on production: `sudo ./scripts/ci/switch-traffic.sh blue instant`
+- View which container images are running: `docker ps --format 'table {{.Names}}\t{{.Image}}'`

docs/PROMPTS.md

@@ -44,22 +44,43 @@ You are a senior software engineer specializsing in NodeJS, Typescript, front en
 - Make no assumptions. 
 - Ask clarifying questions. 
 - Ultrathink 
-- This application is ready to go into production. 
+- Debug why staging and production websites dont' match even though the docker image ID's match
 - Analysis needs to be done on the CI/CD pipeline 
 
 *** CONTEXT ***
 - Read README.md CLAUDE.md and AI-INDEX.md and follow relevant instructions to understand this code repository in the context of this change. 
-- The current deployment does not take into account no downtime or miniimal downtime updates.
+- The staging site runs on staging.motovaultpro.com and production runs on motovaultpro.com
-- The same runner's build the software that run the software
+- These sites are local so use an MCP that will work with local sites to gather a snapshot.
-- There needs to be a balance of uptime and complexity
+- Example: Staging has the correct title in About Us "Built by enthusiasts. Made for your collection."
-- production will run on a single server to start
+- Exaxmple: Production has the old title in About us "Overall, our goal is to meet each individual's needs with quality, passion, and professionalism."
 
 *** ACTION - CHANGES TO IMPLEMENT ***
 - Research this code base and ask iterative questions to compile a complete plan.
 - We will pair plan this. Ask me for options for various levels of redundancy and automation
 
 
+*** STAGING CONTAINER IMAGES ***
+egullickson@mvp-build:~$ sudo docker image ls
+                                                                                                                                                                                                                  i Info →   U  In Use
+IMAGE                                                         ID             DISK USAGE   CONTENT SIZE   EXTRA
+git.motovaultpro.com/egullickson/backend:3321d82              67b2480ddac5        485MB         76.3MB    U
+git.motovaultpro.com/egullickson/frontend:3321d82             e3e1ee18df42       96.1MB         28.7MB    U
+git.motovaultpro.com/egullickson/mirrors/postgres:18-alpine   6723ec6d445f        402MB          112MB    U
+git.motovaultpro.com/egullickson/mirrors/redis:8.4-alpine     8360960f5fb5        130MB         33.4MB    U
+git.motovaultpro.com/egullickson/mirrors/traefik:v3.6         13e903c820df        239MB           52MB    U
+egullickson@mvp-build:~$
 
+*** PRODUCTION CONTAINER IMAGES ***
+
+egullickson@mvp-prod:~$ sudo docker image ls
+                                                                                                                                                                                                                  i Info →   U  In Use
+IMAGE                                                         ID             DISK USAGE   CONTENT SIZE   EXTRA
+git.motovaultpro.com/egullickson/backend:latest               67b2480ddac5        485MB         76.3MB    U
+git.motovaultpro.com/egullickson/frontend:latest              e3e1ee18df42       96.1MB         28.7MB    U
+git.motovaultpro.com/egullickson/mirrors/postgres:18-alpine   6723ec6d445f        402MB          112MB    U
+git.motovaultpro.com/egullickson/mirrors/redis:8.4-alpine     8360960f5fb5        130MB         33.4MB    U
+git.motovaultpro.com/egullickson/mirrors/traefik:v3.6         13e903c820df        239MB           52MB    U
+egullickson@mvp-prod:~$
 
 
 

scripts/ci/switch-traffic.sh

@@ -57,18 +57,19 @@ update_weights() {
 
   echo "  Setting weights: blue=$blue_weight, green=$green_weight"
 
-  # Use sed to update weights in the YAML file
+  # YAML structure has name and weight on separate lines:
-  # Frontend blue weight
+  #   - name: mvp-frontend-blue-svc
-  sed -i.bak -E "s/(name: mvp-frontend-blue-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $blue_weight/" "$TRAEFIK_CONFIG"
+  #     weight: 100
-  # Frontend green weight
+  # Use awk for reliable multi-line pattern matching (more portable than sed)
-  sed -i.bak -E "s/(name: mvp-frontend-green-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $green_weight/" "$TRAEFIK_CONFIG"
+  # Match "- name: service-name" to avoid matching the loadBalancer section
-  # Backend blue weight
-  sed -i.bak -E "s/(name: mvp-backend-blue-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $blue_weight/" "$TRAEFIK_CONFIG"
-  # Backend green weight
-  sed -i.bak -E "s/(name: mvp-backend-green-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $green_weight/" "$TRAEFIK_CONFIG"
 
-  # Clean up backup files
+  awk -v blue="$blue_weight" -v green="$green_weight" '
-  rm -f "${TRAEFIK_CONFIG}.bak"
+    /- name: mvp-frontend-blue-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " blue); print; next}
+    /- name: mvp-frontend-green-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " green); print; next}
+    /- name: mvp-backend-blue-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " blue); print; next}
+    /- name: mvp-backend-green-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " green); print; next}
+    {print}
+  ' "$TRAEFIK_CONFIG" > "${TRAEFIK_CONFIG}.tmp" && mv "${TRAEFIK_CONFIG}.tmp" "$TRAEFIK_CONFIG"
 
   # Traefik watches the file and reloads automatically
   # Give it a moment to pick up changes