fix: CI/CD blue-green deployment path bug causing stale production content
All checks were successful
Deploy to Staging / Build Images (push) Successful in 23s
Deploy to Staging / Deploy to Staging (push) Successful in 26s
Deploy to Staging / Verify Staging (push) Successful in 5s
Deploy to Staging / Notify Staging Ready (push) Successful in 5s
Deploy to Staging / Notify Staging Failure (push) Has been skipped

Root cause: switch-traffic.sh was modifying Traefik config in the CI checkout
directory ($GITHUB_WORKSPACE) instead of the deployment directory ($DEPLOY_PATH).
Additionally, the sed patterns didn't work with multi-line YAML structure.

Changes:
- Add DEPLOY_PATH environment variable support to all CI scripts
- Add --force-recreate flag to ensure containers are recreated with new images
- Fix weight update to use awk for reliable multi-line YAML editing
- Add scripts/ directory to rsync so SREs can run scripts from /opt/motovaultpro
- Add image verification step to confirm containers use expected images
- Add weight verification to confirm Traefik routing was updated
- Add routing validation step to verify traffic switch succeeded

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Eric Gullickson
2025-12-31 10:39:27 -06:00
parent 13abbc16d7
commit c57a05daa5
4 changed files with 46 additions and 18 deletions

View File

@@ -57,18 +57,19 @@ update_weights() {
echo " Setting weights: blue=$blue_weight, green=$green_weight"
# Use sed to update weights in the YAML file
# Frontend blue weight
sed -i.bak -E "s/(name: mvp-frontend-blue-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $blue_weight/" "$TRAEFIK_CONFIG"
# Frontend green weight
sed -i.bak -E "s/(name: mvp-frontend-green-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $green_weight/" "$TRAEFIK_CONFIG"
# Backend blue weight
sed -i.bak -E "s/(name: mvp-backend-blue-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $blue_weight/" "$TRAEFIK_CONFIG"
# Backend green weight
sed -i.bak -E "s/(name: mvp-backend-green-svc[[:space:]]+weight:)[[:space:]]+[0-9]+/\1 $green_weight/" "$TRAEFIK_CONFIG"
# YAML structure has name and weight on separate lines:
# - name: mvp-frontend-blue-svc
# weight: 100
# Use awk for reliable multi-line pattern matching (more portable than sed)
# Match "- name: service-name" to avoid matching the loadBalancer section
# Clean up backup files
rm -f "${TRAEFIK_CONFIG}.bak"
awk -v blue="$blue_weight" -v green="$green_weight" '
/- name: mvp-frontend-blue-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " blue); print; next}
/- name: mvp-frontend-green-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " green); print; next}
/- name: mvp-backend-blue-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " blue); print; next}
/- name: mvp-backend-green-svc$/ {print; getline; sub(/weight: [0-9]+/, "weight: " green); print; next}
{print}
' "$TRAEFIK_CONFIG" > "${TRAEFIK_CONFIG}.tmp" && mv "${TRAEFIK_CONFIG}.tmp" "$TRAEFIK_CONFIG"
# Traefik watches the file and reloads automatically
# Give it a moment to pick up changes