docs: add unified logging system documentation and CI/CD integration (#87) #94
@@ -18,7 +18,7 @@ env:
|
|||||||
COMPOSE_FILE: docker-compose.yml
|
COMPOSE_FILE: docker-compose.yml
|
||||||
COMPOSE_STAGING: docker-compose.staging.yml
|
COMPOSE_STAGING: docker-compose.staging.yml
|
||||||
HEALTH_CHECK_TIMEOUT: "60"
|
HEALTH_CHECK_TIMEOUT: "60"
|
||||||
LOG_LEVEL: INFO
|
LOG_LEVEL: DEBUG
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
# ============================================
|
# ============================================
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# MotoVaultPro
|
# MotoVaultPro
|
||||||
|
|
||||||
Single-tenant vehicle management application with 5-container architecture (Traefik, Frontend, Backend, PostgreSQL, Redis).
|
Single-tenant vehicle management application with 9-container architecture (6 application: Traefik, Frontend, Backend, OCR, PostgreSQL, Redis + 3 logging: Loki, Promtail, Grafana).
|
||||||
|
|
||||||
## Files
|
## Files
|
||||||
|
|
||||||
@@ -172,8 +172,8 @@ Issues are the source of truth. See `.ai/workflow-contract.json` for complete wo
|
|||||||
|
|
||||||
## Architecture Context for AI
|
## Architecture Context for AI
|
||||||
|
|
||||||
### Simplified 5-Container Architecture
|
### 9-Container Architecture
|
||||||
**MotoVaultPro uses a simplified architecture:** A single-tenant application with 5 containers - Traefik, Frontend, Backend, PostgreSQL, and Redis. Application features in `backend/src/features/[name]/` are self-contained modules within the backend service, including the platform feature for vehicle data and VIN decoding.
|
**MotoVaultPro uses a unified architecture:** A single-tenant application with 9 containers - 6 application (Traefik, Frontend, Backend, OCR, PostgreSQL, Redis) + 3 logging (Loki, Promtail, Grafana). Application features in `backend/src/features/[name]/` are self-contained modules within the backend service, including the platform feature for vehicle data and VIN decoding. See `docs/LOGGING.md` for unified logging system documentation.
|
||||||
|
|
||||||
### Key Principles for AI Understanding
|
### Key Principles for AI Understanding
|
||||||
- **Feature Capsule Organization**: Application features are self-contained modules within the backend
|
- **Feature Capsule Organization**: Application features are self-contained modules within the backend
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# MotoVaultPro — Simplified Architecture
|
# MotoVaultPro — Simplified Architecture
|
||||||
|
|
||||||
Simplified 5-container architecture with integrated platform feature.
|
9-container architecture (6 application + 3 logging) with integrated platform feature.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
- Mobile + Desktop: Implement and test every feature on both.
|
- Mobile + Desktop: Implement and test every feature on both.
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
| File | What | When to read |
|
| File | What | When to read |
|
||||||
| ---- | ---- | ------------ |
|
| ---- | ---- | ------------ |
|
||||||
| `README.md` | Documentation hub and navigation | Starting point for any docs |
|
| `README.md` | Documentation hub and navigation | Starting point for any docs |
|
||||||
| `PLATFORM-SERVICES.md` | 5-container architecture overview | Understanding system topology |
|
| `PLATFORM-SERVICES.md` | 9-container architecture overview | Understanding system topology |
|
||||||
| `ARCHITECTURE-OVERVIEW.md` | Detailed architecture documentation | Deep dive into system design |
|
| `ARCHITECTURE-OVERVIEW.md` | Detailed architecture documentation | Deep dive into system design |
|
||||||
| `DATABASE-SCHEMA.md` | Database table definitions | Schema changes, data modeling |
|
| `DATABASE-SCHEMA.md` | Database table definitions | Schema changes, data modeling |
|
||||||
| `DATABASE-MIGRATION.md` | Migration procedures | Running or writing migrations |
|
| `DATABASE-MIGRATION.md` | Migration procedures | Running or writing migrations |
|
||||||
@@ -17,5 +17,5 @@
|
|||||||
| `BUILD-SERVER-SETUP.md` | Build server setup | Infrastructure setup |
|
| `BUILD-SERVER-SETUP.md` | Build server setup | Infrastructure setup |
|
||||||
| `AUDIT.md` | Audit documentation | Security audits, compliance |
|
| `AUDIT.md` | Audit documentation | Security audits, compliance |
|
||||||
| `MVP-COLOR-SCHEME.md` | Color scheme reference | UI styling decisions |
|
| `MVP-COLOR-SCHEME.md` | Color scheme reference | UI styling decisions |
|
||||||
| `UX-DEBUGGING.md` | UX debugging techniques | Debugging UI issues |
|
| `LOGGING.md` | Unified logging system | Log levels, correlation IDs, Grafana |
|
||||||
|
|
||||||
|
|||||||
118
docs/LOGGING.md
Normal file
118
docs/LOGGING.md
Normal file
@@ -0,0 +1,118 @@
|
|||||||
|
# Unified Logging System
|
||||||
|
|
||||||
|
MotoVaultPro uses a unified logging system with centralized log aggregation.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
- **Single Control**: One `LOG_LEVEL` environment variable controls all containers
|
||||||
|
- **Correlation IDs**: `requestId` field traces requests across services
|
||||||
|
- **Centralized Aggregation**: Grafana + Loki for log querying and visualization
|
||||||
|
|
||||||
|
## LOG_LEVEL Values
|
||||||
|
|
||||||
|
| Level | Frontend | Backend | PostgreSQL | Redis | Traefik |
|
||||||
|
|-------|----------|---------|------------|-------|---------|
|
||||||
|
| DEBUG | debug | debug | all queries, 0ms | debug | DEBUG |
|
||||||
|
| INFO | info | info | DDL only, 500ms | verbose | INFO |
|
||||||
|
| WARN | warn | warn | errors, 1000ms | notice | WARN |
|
||||||
|
| ERROR | error | error | errors only | warning | ERROR |
|
||||||
|
|
||||||
|
## Environment Defaults
|
||||||
|
|
||||||
|
| Environment | LOG_LEVEL | Purpose |
|
||||||
|
|-------------|-----------|---------|
|
||||||
|
| Development | DEBUG | Full debugging locally |
|
||||||
|
| Staging | DEBUG | Full debugging in staging |
|
||||||
|
| Production | INFO | Standard production logging |
|
||||||
|
|
||||||
|
## Correlation IDs
|
||||||
|
|
||||||
|
All logs include a `requestId` field (UUID v4) for tracing requests:
|
||||||
|
|
||||||
|
- **Traefik**: Forwards X-Request-Id if present
|
||||||
|
- **Backend**: Generates UUID if X-Request-Id missing, includes in all logs
|
||||||
|
- **Frontend**: Includes requestId in API call logs
|
||||||
|
|
||||||
|
### Example Log Entry
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"level": "info",
|
||||||
|
"time": "2024-01-15T10:30:00.000Z",
|
||||||
|
"requestId": "550e8400-e29b-41d4-a716-446655440000",
|
||||||
|
"msg": "Request processed",
|
||||||
|
"method": "GET",
|
||||||
|
"path": "/api/vehicles",
|
||||||
|
"status": 200,
|
||||||
|
"duration": 45
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Grafana Access
|
||||||
|
|
||||||
|
- URL: https://logs.motovaultpro.com
|
||||||
|
- Default credentials: admin/admin (change on first login)
|
||||||
|
|
||||||
|
### Example LogQL Queries
|
||||||
|
|
||||||
|
Query by requestId:
|
||||||
|
```
|
||||||
|
{container="mvp-backend"} |= "550e8400-e29b-41d4"
|
||||||
|
```
|
||||||
|
|
||||||
|
Query all errors:
|
||||||
|
```
|
||||||
|
{container=~"mvp-.*"} | json | level="error"
|
||||||
|
```
|
||||||
|
|
||||||
|
Query slow requests (>500ms):
|
||||||
|
```
|
||||||
|
{container="mvp-backend"} | json | duration > 500
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Logging configuration is generated by `scripts/ci/generate-log-config.sh`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Generate DEBUG level config
|
||||||
|
./scripts/ci/generate-log-config.sh DEBUG
|
||||||
|
|
||||||
|
# Generate INFO level config
|
||||||
|
./scripts/ci/generate-log-config.sh INFO
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates `.env.logging` which is sourced by docker-compose.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
+-----------------------------------------------------------------------+
|
||||||
|
| CI/CD PIPELINE |
|
||||||
|
| LOG_LEVEL --> generate-log-config.sh --> .env.logging |
|
||||||
|
+-----------------------------------------------------------------------+
|
||||||
|
|
|
||||||
|
v
|
||||||
|
+-----------------------------------------------------------------------+
|
||||||
|
| APPLICATION LAYER |
|
||||||
|
| Frontend Backend OCR Postgres Redis Traefik |
|
||||||
|
| | | | | | | |
|
||||||
|
| +---------+---------+---------+---------+---------+ |
|
||||||
|
| | |
|
||||||
|
| Docker Log Driver (json-file, 10m x 3) |
|
||||||
|
+-----------------------------------------------------------------------+
|
||||||
|
|
|
||||||
|
v
|
||||||
|
Promtail --> Loki (30-day retention) --> Grafana
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Logs not appearing in Grafana
|
||||||
|
|
||||||
|
1. Check Promtail is running: `docker logs mvp-promtail`
|
||||||
|
2. Check Loki is healthy: `curl http://localhost:3100/ready`
|
||||||
|
3. Verify log rotation is not too aggressive
|
||||||
|
|
||||||
|
### Invalid LOG_LEVEL
|
||||||
|
|
||||||
|
Both frontend and backend will warn and fall back to 'info' if an invalid LOG_LEVEL is provided.
|
||||||
@@ -54,7 +54,9 @@ VIN decoding is planned but not yet implemented. Future capabilities will includ
|
|||||||
The platform module deploys with the main application:
|
The platform module deploys with the main application:
|
||||||
- Same deployment pipeline
|
- Same deployment pipeline
|
||||||
- Shares database and cache
|
- Shares database and cache
|
||||||
- Deployed as part of the five-container stack
|
- Deployed as part of the 9-container stack (6 application + 3 logging)
|
||||||
|
|
||||||
|
See `docs/LOGGING.md` for unified logging system documentation.
|
||||||
|
|
||||||
## Integration Patterns
|
## Integration Patterns
|
||||||
|
|
||||||
|
|||||||
@@ -1,9 +1,10 @@
|
|||||||
# MotoVaultPro Documentation
|
# MotoVaultPro Documentation
|
||||||
|
|
||||||
Project documentation hub for the 5-container single-tenant architecture with integrated platform feature.
|
Project documentation hub for the 9-container single-tenant architecture (6 application + 3 logging) with integrated platform feature.
|
||||||
|
|
||||||
## Navigation
|
## Navigation
|
||||||
|
|
||||||
|
- Logging: `docs/LOGGING.md`
|
||||||
- Architecture: `docs/PLATFORM-SERVICES.md`
|
- Architecture: `docs/PLATFORM-SERVICES.md`
|
||||||
- Security: `docs/SECURITY.md`
|
- Security: `docs/SECURITY.md`
|
||||||
- Vehicles API (authoritative): `docs/VEHICLES-API.md`
|
- Vehicles API (authoritative): `docs/VEHICLES-API.md`
|
||||||
|
|||||||
@@ -1,89 +0,0 @@
|
|||||||
# MotoVaultPro Debug Console Configuration
|
|
||||||
|
|
||||||
## CRITICAL: Console Logs Are Stripped in Production Builds
|
|
||||||
|
|
||||||
**Before debugging any UX/UI issues, ALWAYS enable console logging first.**
|
|
||||||
|
|
||||||
## Production Build Console Stripping
|
|
||||||
|
|
||||||
The Vite build configuration in `frontend/vite.config.ts` aggressively removes ALL console statements in production:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Lines 60-62: Terser removes console logs
|
|
||||||
terserOptions: {
|
|
||||||
compress: {
|
|
||||||
drop_console: true, // ← This removes ALL console.log statements
|
|
||||||
drop_debugger: true,
|
|
||||||
pure_funcs: ['console.log', 'console.info', 'console.debug'],
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Line 74: ESBuild also removes console logs
|
|
||||||
esbuild: {
|
|
||||||
drop: ['console', 'debugger'], // ← Additional console removal
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Debug Protocol for UX Issues
|
|
||||||
|
|
||||||
When debugging **any** UX/UI problems (buttons not working, state not updating, components not rendering):
|
|
||||||
|
|
||||||
### 1. Enable Console Logging FIRST
|
|
||||||
```typescript
|
|
||||||
// In frontend/vite.config.ts
|
|
||||||
|
|
||||||
// TEMPORARILY change these lines:
|
|
||||||
drop_console: false, // Keep console logs for debugging
|
|
||||||
// pure_funcs: ['console.log', 'console.info', 'console.debug'], // Comment out
|
|
||||||
|
|
||||||
// AND:
|
|
||||||
drop: ['debugger'], // Keep console, only drop debugger
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Add Debug Statements
|
|
||||||
```typescript
|
|
||||||
// Example debug patterns:
|
|
||||||
console.log('[DEBUG] Component render - state:', someState);
|
|
||||||
console.log('[DEBUG] useEffect triggered - deps:', dep1, dep2);
|
|
||||||
console.log('[DEBUG] Button clicked - current state:', state);
|
|
||||||
console.log('[DEBUG] Store action called:', actionName, payload);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Rebuild and Test
|
|
||||||
```bash
|
|
||||||
make rebuild # Required to apply vite.config.ts changes
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Fix the Issue
|
|
||||||
Use browser dev tools console output to identify the root cause.
|
|
||||||
|
|
||||||
### 5. Clean Up and Restore Production Settings
|
|
||||||
```typescript
|
|
||||||
// Restore production console stripping:
|
|
||||||
drop_console: true, // Remove console logs in production
|
|
||||||
pure_funcs: ['console.log', 'console.info', 'console.debug'],
|
|
||||||
drop: ['console', 'debugger'], // Additional cleanup
|
|
||||||
```
|
|
||||||
|
|
||||||
Remove debug console.log statements and rebuild.
|
|
||||||
|
|
||||||
## Common UX Issue Patterns
|
|
||||||
|
|
||||||
1. **Buttons not working**: Usually state management or event handler issues
|
|
||||||
2. **Components not re-rendering**: Missing dependencies in hooks or store subscription problems
|
|
||||||
3. **useEffect fighting with user actions**: Dependencies causing infinite loops
|
|
||||||
4. **Store state not updating**: Action functions not properly bound or called
|
|
||||||
|
|
||||||
## Example: The Sidebar Issue
|
|
||||||
|
|
||||||
The sidebar X button wasn't working because:
|
|
||||||
- `useEffect` dependency array included `sidebarOpen`
|
|
||||||
- When user clicked X → `sidebarOpen` became `false`
|
|
||||||
- `useEffect` fired → immediately called `setSidebarOpen(true)`
|
|
||||||
- User action was overridden by the `useEffect`
|
|
||||||
|
|
||||||
**Without console debugging enabled, this was invisible!**
|
|
||||||
|
|
||||||
## Key Reminder
|
|
||||||
|
|
||||||
**Never assume JavaScript is working correctly in production builds without console debugging enabled first.** The aggressive console stripping makes silent failures very common.
|
|
||||||
Reference in New Issue
Block a user