From 72275096f828bb5ecb903d4820cbaa3f4883dd36 Mon Sep 17 00:00:00 2001 From: Eric Gullickson <16152721+ericgullickson@users.noreply.github.com> Date: Wed, 4 Feb 2026 20:50:20 -0600 Subject: [PATCH] docs: add unified logging system documentation and CI/CD integration (refs #87) - Update staging workflow to use LOG_LEVEL=DEBUG - Create docs/LOGGING.md with unified logging documentation - Delete docs/UX-DEBUGGING.md (replaced by LOGGING.md) - Update architecture to 9-container (6 app + 3 logging) - Update CLAUDE.md, README.md, docs/README.md, docs/CLAUDE.md - Update docs/PLATFORM-SERVICES.md deployment section Co-Authored-By: Claude Opus 4.5 --- .gitea/workflows/staging.yaml | 2 +- CLAUDE.md | 6 +- README.md | 2 +- docs/CLAUDE.md | 4 +- docs/LOGGING.md | 118 ++++++++++++++++++++++++++++++++++ docs/PLATFORM-SERVICES.md | 4 +- docs/README.md | 3 +- docs/UX-DEBUGGING.md | 89 ------------------------- 8 files changed, 130 insertions(+), 98 deletions(-) create mode 100644 docs/LOGGING.md delete mode 100644 docs/UX-DEBUGGING.md diff --git a/.gitea/workflows/staging.yaml b/.gitea/workflows/staging.yaml index c6bb4e5..1886b3b 100644 --- a/.gitea/workflows/staging.yaml +++ b/.gitea/workflows/staging.yaml @@ -18,7 +18,7 @@ env: COMPOSE_FILE: docker-compose.yml COMPOSE_STAGING: docker-compose.staging.yml HEALTH_CHECK_TIMEOUT: "60" - LOG_LEVEL: INFO + LOG_LEVEL: DEBUG jobs: # ============================================ diff --git a/CLAUDE.md b/CLAUDE.md index 81cb2af..592ada1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # MotoVaultPro -Single-tenant vehicle management application with 5-container architecture (Traefik, Frontend, Backend, PostgreSQL, Redis). +Single-tenant vehicle management application with 9-container architecture (6 application: Traefik, Frontend, Backend, OCR, PostgreSQL, Redis + 3 logging: Loki, Promtail, Grafana). ## Files @@ -172,8 +172,8 @@ Issues are the source of truth. See `.ai/workflow-contract.json` for complete wo ## Architecture Context for AI -### Simplified 5-Container Architecture -**MotoVaultPro uses a simplified architecture:** A single-tenant application with 5 containers - Traefik, Frontend, Backend, PostgreSQL, and Redis. Application features in `backend/src/features/[name]/` are self-contained modules within the backend service, including the platform feature for vehicle data and VIN decoding. +### 9-Container Architecture +**MotoVaultPro uses a unified architecture:** A single-tenant application with 9 containers - 6 application (Traefik, Frontend, Backend, OCR, PostgreSQL, Redis) + 3 logging (Loki, Promtail, Grafana). Application features in `backend/src/features/[name]/` are self-contained modules within the backend service, including the platform feature for vehicle data and VIN decoding. See `docs/LOGGING.md` for unified logging system documentation. ### Key Principles for AI Understanding - **Feature Capsule Organization**: Application features are self-contained modules within the backend diff --git a/README.md b/README.md index 68957f2..f138378 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # MotoVaultPro — Simplified Architecture -Simplified 5-container architecture with integrated platform feature. +9-container architecture (6 application + 3 logging) with integrated platform feature. ## Requirements - Mobile + Desktop: Implement and test every feature on both. diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md index 0a33a36..a0e2637 100644 --- a/docs/CLAUDE.md +++ b/docs/CLAUDE.md @@ -5,7 +5,7 @@ | File | What | When to read | | ---- | ---- | ------------ | | `README.md` | Documentation hub and navigation | Starting point for any docs | -| `PLATFORM-SERVICES.md` | 5-container architecture overview | Understanding system topology | +| `PLATFORM-SERVICES.md` | 9-container architecture overview | Understanding system topology | | `ARCHITECTURE-OVERVIEW.md` | Detailed architecture documentation | Deep dive into system design | | `DATABASE-SCHEMA.md` | Database table definitions | Schema changes, data modeling | | `DATABASE-MIGRATION.md` | Migration procedures | Running or writing migrations | @@ -17,5 +17,5 @@ | `BUILD-SERVER-SETUP.md` | Build server setup | Infrastructure setup | | `AUDIT.md` | Audit documentation | Security audits, compliance | | `MVP-COLOR-SCHEME.md` | Color scheme reference | UI styling decisions | -| `UX-DEBUGGING.md` | UX debugging techniques | Debugging UI issues | +| `LOGGING.md` | Unified logging system | Log levels, correlation IDs, Grafana | diff --git a/docs/LOGGING.md b/docs/LOGGING.md new file mode 100644 index 0000000..0cfffff --- /dev/null +++ b/docs/LOGGING.md @@ -0,0 +1,118 @@ +# Unified Logging System + +MotoVaultPro uses a unified logging system with centralized log aggregation. + +## Overview + +- **Single Control**: One `LOG_LEVEL` environment variable controls all containers +- **Correlation IDs**: `requestId` field traces requests across services +- **Centralized Aggregation**: Grafana + Loki for log querying and visualization + +## LOG_LEVEL Values + +| Level | Frontend | Backend | PostgreSQL | Redis | Traefik | +|-------|----------|---------|------------|-------|---------| +| DEBUG | debug | debug | all queries, 0ms | debug | DEBUG | +| INFO | info | info | DDL only, 500ms | verbose | INFO | +| WARN | warn | warn | errors, 1000ms | notice | WARN | +| ERROR | error | error | errors only | warning | ERROR | + +## Environment Defaults + +| Environment | LOG_LEVEL | Purpose | +|-------------|-----------|---------| +| Development | DEBUG | Full debugging locally | +| Staging | DEBUG | Full debugging in staging | +| Production | INFO | Standard production logging | + +## Correlation IDs + +All logs include a `requestId` field (UUID v4) for tracing requests: + +- **Traefik**: Forwards X-Request-Id if present +- **Backend**: Generates UUID if X-Request-Id missing, includes in all logs +- **Frontend**: Includes requestId in API call logs + +### Example Log Entry +```json +{ + "level": "info", + "time": "2024-01-15T10:30:00.000Z", + "requestId": "550e8400-e29b-41d4-a716-446655440000", + "msg": "Request processed", + "method": "GET", + "path": "/api/vehicles", + "status": 200, + "duration": 45 +} +``` + +## Grafana Access + +- URL: https://logs.motovaultpro.com +- Default credentials: admin/admin (change on first login) + +### Example LogQL Queries + +Query by requestId: +``` +{container="mvp-backend"} |= "550e8400-e29b-41d4" +``` + +Query all errors: +``` +{container=~"mvp-.*"} | json | level="error" +``` + +Query slow requests (>500ms): +``` +{container="mvp-backend"} | json | duration > 500 +``` + +## Configuration + +Logging configuration is generated by `scripts/ci/generate-log-config.sh`: + +```bash +# Generate DEBUG level config +./scripts/ci/generate-log-config.sh DEBUG + +# Generate INFO level config +./scripts/ci/generate-log-config.sh INFO +``` + +This creates `.env.logging` which is sourced by docker-compose. + +## Architecture + +``` ++-----------------------------------------------------------------------+ +| CI/CD PIPELINE | +| LOG_LEVEL --> generate-log-config.sh --> .env.logging | ++-----------------------------------------------------------------------+ + | + v ++-----------------------------------------------------------------------+ +| APPLICATION LAYER | +| Frontend Backend OCR Postgres Redis Traefik | +| | | | | | | | +| +---------+---------+---------+---------+---------+ | +| | | +| Docker Log Driver (json-file, 10m x 3) | ++-----------------------------------------------------------------------+ + | + v + Promtail --> Loki (30-day retention) --> Grafana +``` + +## Troubleshooting + +### Logs not appearing in Grafana + +1. Check Promtail is running: `docker logs mvp-promtail` +2. Check Loki is healthy: `curl http://localhost:3100/ready` +3. Verify log rotation is not too aggressive + +### Invalid LOG_LEVEL + +Both frontend and backend will warn and fall back to 'info' if an invalid LOG_LEVEL is provided. diff --git a/docs/PLATFORM-SERVICES.md b/docs/PLATFORM-SERVICES.md index 13b35ed..8fe3686 100644 --- a/docs/PLATFORM-SERVICES.md +++ b/docs/PLATFORM-SERVICES.md @@ -54,7 +54,9 @@ VIN decoding is planned but not yet implemented. Future capabilities will includ The platform module deploys with the main application: - Same deployment pipeline - Shares database and cache -- Deployed as part of the five-container stack +- Deployed as part of the 9-container stack (6 application + 3 logging) + +See `docs/LOGGING.md` for unified logging system documentation. ## Integration Patterns diff --git a/docs/README.md b/docs/README.md index 7af7a66..f61f945 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,9 +1,10 @@ # MotoVaultPro Documentation -Project documentation hub for the 5-container single-tenant architecture with integrated platform feature. +Project documentation hub for the 9-container single-tenant architecture (6 application + 3 logging) with integrated platform feature. ## Navigation +- Logging: `docs/LOGGING.md` - Architecture: `docs/PLATFORM-SERVICES.md` - Security: `docs/SECURITY.md` - Vehicles API (authoritative): `docs/VEHICLES-API.md` diff --git a/docs/UX-DEBUGGING.md b/docs/UX-DEBUGGING.md deleted file mode 100644 index 14c0c78..0000000 --- a/docs/UX-DEBUGGING.md +++ /dev/null @@ -1,89 +0,0 @@ -# MotoVaultPro Debug Console Configuration - -## CRITICAL: Console Logs Are Stripped in Production Builds - -**Before debugging any UX/UI issues, ALWAYS enable console logging first.** - -## Production Build Console Stripping - -The Vite build configuration in `frontend/vite.config.ts` aggressively removes ALL console statements in production: - -```typescript -// Lines 60-62: Terser removes console logs -terserOptions: { - compress: { - drop_console: true, // ← This removes ALL console.log statements - drop_debugger: true, - pure_funcs: ['console.log', 'console.info', 'console.debug'], - } -} - -// Line 74: ESBuild also removes console logs -esbuild: { - drop: ['console', 'debugger'], // ← Additional console removal -} -``` - -## Debug Protocol for UX Issues - -When debugging **any** UX/UI problems (buttons not working, state not updating, components not rendering): - -### 1. Enable Console Logging FIRST -```typescript -// In frontend/vite.config.ts - -// TEMPORARILY change these lines: -drop_console: false, // Keep console logs for debugging -// pure_funcs: ['console.log', 'console.info', 'console.debug'], // Comment out - -// AND: -drop: ['debugger'], // Keep console, only drop debugger -``` - -### 2. Add Debug Statements -```typescript -// Example debug patterns: -console.log('[DEBUG] Component render - state:', someState); -console.log('[DEBUG] useEffect triggered - deps:', dep1, dep2); -console.log('[DEBUG] Button clicked - current state:', state); -console.log('[DEBUG] Store action called:', actionName, payload); -``` - -### 3. Rebuild and Test -```bash -make rebuild # Required to apply vite.config.ts changes -``` - -### 4. Fix the Issue -Use browser dev tools console output to identify the root cause. - -### 5. Clean Up and Restore Production Settings -```typescript -// Restore production console stripping: -drop_console: true, // Remove console logs in production -pure_funcs: ['console.log', 'console.info', 'console.debug'], -drop: ['console', 'debugger'], // Additional cleanup -``` - -Remove debug console.log statements and rebuild. - -## Common UX Issue Patterns - -1. **Buttons not working**: Usually state management or event handler issues -2. **Components not re-rendering**: Missing dependencies in hooks or store subscription problems -3. **useEffect fighting with user actions**: Dependencies causing infinite loops -4. **Store state not updating**: Action functions not properly bound or called - -## Example: The Sidebar Issue - -The sidebar X button wasn't working because: -- `useEffect` dependency array included `sidebarOpen` -- When user clicked X → `sidebarOpen` became `false` -- `useEffect` fired → immediately called `setSidebarOpen(true)` -- User action was overridden by the `useEffect` - -**Without console debugging enabled, this was invisible!** - -## Key Reminder - -**Never assume JavaScript is working correctly in production builds without console debugging enabled first.** The aggressive console stripping makes silent failures very common. \ No newline at end of file