motovaultpro/docs/changes/DOCUMENTS.md

Documents Feature Plan (S3-Compatible, Phased)

This plan aligns with the current codebase: MinIO is running (admin-minio), object storage credentials are mounted as secrets, and appConfig.getMinioConfig() is available. We will implement a generic S3-compatible storage surface with a MinIO-backed adapter first, following the Docker‑first, production‑only workflow and mobile+desktop requirements.

— Read me quick —

• Storage: Start with MinIO SDK via getMinioConfig(). Keep the interface S3‑generic to support AWS S3 later without changing features.
• Auth/Tenant: All endpoints use [fastify.authenticate, tenantMiddleware].
• Testing: Use Jest; run via containers with make test.
• Mobile+Desktop: Follow existing Zustand nav, React Router routes, GlassCard components, and React Query offlineFirst.

Handoff markers are provided at the end of each phase. If work pauses, pick up from the next “Done when” checklist.

Phase 0 — Baseline Verification

Objectives

• Confirm configuration and dependencies to avoid rework.

Tasks

• Verify MinIO configuration in config/app/production.yml → minio.endpoint, minio.port, minio.bucket.
• Verify mounted secrets exist for MinIO (secrets/app/minio-access-key.txt, secrets/app/minio-secret-key.txt).
• Verify backend dependency presence:
  • Present: minio@^7.1.3
  • Missing: @fastify/multipart (add to backend/package.json)
• Rebuild and tail logs
  • make rebuild
  • make logs

Done when

• Containers start cleanly and backend logs show no missing module errors.

Status

• MinIO configuration verified in repo (endpoint/port/bucket present) ✓
• MinIO secrets present in repo (mounted paths defined) ✓
• Package check: minio present ✓, @fastify/multipart added to backend/package.json ✓
• Rebuild/logs runtime verification: pending (perform via make rebuild && make logs)

Phase 1 — Storage Foundation (S3-Compatible, MinIO-Backed)

Objectives

• Create a generic storage façade used by features; implement first adapter using MinIO SDK.

Design

• Interface StorageService methods:
  • putObject(bucket, key, bodyOrStream, contentType, metadata?)
  • getObjectStream(bucket, key)
  • deleteObject(bucket, key)
  • headObject(bucket, key)
  • getSignedUrl(bucket, key, { method: 'GET'|'PUT', expiresSeconds })
• Key scheme: documents/{userId}/{vehicleId}/{documentId}/{version}/{uuid}.{ext}
• Security: Private objects only; short‑lived signed URLs when needed.

Files

• backend/src/core/storage/storage.service.ts — façade and factory.
• backend/src/core/storage/adapters/minio.adapter.ts — uses MinIO SDK and appConfig.getMinioConfig().

Tasks

• Implement MinIO client using endpoint/port/accessKey/secretKey/bucket from appConfig.getMinioConfig().
• Ensure streaming APIs are used for uploads/downloads.
• Implement signed URL generation for downloads with short TTL (e.g., 60–300s).

Done when

• Service can put/head/get/delete and generate signed URLs against admin-minio bucket from inside the backend container.

Status

• Storage facade added: backend/src/core/storage/storage.service.ts ✓
• MinIO adapter implemented: backend/src/core/storage/adapters/minio.adapter.ts ✓
• Runtime validation against MinIO: pending (validate post-rebuild) ☐

Phase 2 — Backend HTTP Foundation

Objectives

• Enable file uploads and wire security.

Tasks

• Add @fastify/multipart to backend/package.json.
• In backend/src/app.ts, register multipart with config‑based limits:
  • limits.fileSize sourced from appConfig.config.performance.max_request_size.
• Confirm authentication plugin and tenant middleware are active (already implemented).

Done when

• Backend accepts multipart requests and enforces size limits without errors.

Status

• Dependency added: @fastify/multipart ✓
• Registered in backend/src/app.ts with byte-limit parser ✓
• Runtime verification via container: pending ☐

Phase 3 — Documents Feature Capsule (Backend)

Objectives

• Create the feature capsule with schema, repository, service, routes, and validators, following existing patterns (see vehicles and fuel‑logs).

Structure (backend)

backend/src/features/documents/
├── README.md
├── index.ts
├── api/
│   ├── documents.routes.ts
│   ├── documents.controller.ts
│   └── documents.validation.ts
├── domain/
│   ├── documents.service.ts
│   └── documents.types.ts
├── data/
│   └── documents.repository.ts
├── migrations/
│   └── 001_create_documents_table.sql
└── tests/
    ├── unit/
    └── integration/

Database schema

• Table documents:
  • id UUID PK
  • user_id VARCHAR(255)
  • vehicle_id UUID FK → vehicles(id)
  • document_type VARCHAR(32) CHECK IN ('insurance','registration')
  • title VARCHAR(200); notes TEXT NULL; details JSONB
  • storage_bucket VARCHAR(128); storage_key VARCHAR(512)
  • file_name VARCHAR(255); content_type VARCHAR(128); file_size BIGINT; file_hash VARCHAR(128) NULL
  • issued_date DATE NULL; expiration_date DATE NULL
  • created_at TIMESTAMP DEFAULT now(); updated_at TIMESTAMP DEFAULT now() with update_updated_at_column() trigger
  • deleted_at TIMESTAMP NULL
• Indexes: (user_id), (vehicle_id), (user_id, vehicle_id), (document_type), (expiration_date); optional GIN on details if needed.

API endpoints

POST   /api/documents                      # Create metadata (with/without file)
GET    /api/documents                      # List (filters: vehicleId, type, expiresBefore)
GET    /api/documents/:id                  # Get metadata
PUT    /api/documents/:id                  # Update metadata/details
DELETE /api/documents/:id                  # Soft delete (and delete object)
GET    /api/documents/vehicle/:vehicleId   # List by vehicle
POST   /api/documents/:id/upload           # Upload/replace file (multipart)
GET    /api/documents/:id/download         # Download (proxy stream or signed URL)

• Pre‑handlers: [fastify.authenticate, tenantMiddleware] for all routes.
• Validation: Zod schemas for params/query/body in documents.validation.ts.
• Ownership: Validate vehicle_id belongs to user_id using vehicles pattern (like fuel‑logs).

Wire‑up

• Register in backend/src/app.ts:
  • import { documentsRoutes } from './features/documents/api/documents.routes'
  • await app.register(documentsRoutes, { prefix: '/api' })
• Health: Update /health feature list to include documents.
• Migrations: Add 'features/documents' to MIGRATION_ORDER in backend/src/_system/migrations/run-all.ts after 'features/vehicles'.

Done when

• CRUD + upload/download endpoints are reachable and secured; migrations run in correct order; ownership enforced.

Status

• Capsule scaffolded (api/domain/data/tests/migrations/README) ✓
• Migration added backend/src/features/documents/migrations/001_create_documents_table.sql ✓
• Registered routes in backend/src/app.ts with /api prefix ✓
• Health feature list updated to include documents ✓
• Migration order updated in backend/src/_system/migrations/run-all.ts ✓
• CRUD handlers for metadata implemented ✓
• Upload endpoint implemented with multipart streaming, MIME allowlist, and storage meta update ✓
• Download endpoint implemented with proxy streaming and inline/attachment disposition ✓
• Ownership validation on create via vehicles check ✓
• Runtime verification in container: pending ☐

Phase 4 — Frontend Feature (Mobile + Desktop)

Objectives

• Implement documents UI following existing navigation, layout, and data patterns.

Structure (frontend)

frontend/src/features/documents/
├── pages/
├── components/
├── hooks/
└── types/

Navigation

• Mobile: Add “Documents” to bottom nav (Zustand store in frontend/src/core/store/navigation.ts).
• Desktop: Add routes in frontend/src/App.tsx for list/detail/upload.
• Sub‑screens (mobile): list → detail → upload; wrap content with GlassCard.

Upload UX

• Mobile camera/gallery: <input type="file" accept="image/*" capture="environment" />.
• Desktop drag‑and‑drop with progress.
• Progress tracking: React Query mutation with progress events; optimistic updates and cache invalidation.
• Offline: Use existing React Query offlineFirst config; queue uploads and retry on reconnect.

Viewer

• Inline image/PDF preview; Content-Disposition inline for images/PDF; gestures (pinch/zoom) for mobile images.

Done when

• Users can list, upload, view, and delete documents on both mobile and desktop with responsive UI and progress.

Status

• Add Documents to mobile bottom nav (Zustand): completed ✓
• Add desktop routes in App.tsx (list/detail/upload): completed ✓
• Scaffold pages/components/hooks structure: completed ✓
• Hook list/detail CRUD endpoints with React Query: completed ✓
• Implement upload with progress UI: completed ✓ (hooks with onUploadProgress; UI in mobile/detail)
• Optimistic updates: partial (invalidate queries on success) ◐
• Offline queuing/retry via React Query networkMode: configured via hooks ✓
• Previews: basic image/PDF preview implemented ✓ (DocumentPreview)
• Gesture-friendly viewer: pending ☐
• Desktop navigation: sidebar now defaults open and includes Documents ✓
• Build hygiene: resolved TS unused import error in frontend documents hooks ✓

Phase 5 — Security, Validation, and Policies

Objectives

• Enforce safe file handling and consistent deletion semantics.

Tasks

• MIME allowlist: application/pdf, image/jpeg, image/png; reject executables.
• Upload size: Enforce via multipart limit tied to performance.max_request_size.
• Deletion: Soft delete DB first; delete object after. Consider retention policy later if required.
• Logging: Create/update/delete/upload/download events include user_id, document_id, vehicle_id (use existing logger).
• Optional rate limiting for upload route (defer dependency until needed).

Done when

• Unsafe files rejected; logs record document events; deletions are consistent.

Status

• MIME allowlist enforced for uploads (PDF, JPEG, PNG) ✓
• Upload size enforced via multipart limit (config-driven) ✓
• Deletion semantics: DB soft-delete and best-effort storage object deletion ✓
• Event logging for document actions: pending ☐

Phase 6 — Testing (Docker-First)

Objectives

• Achieve green tests and linting across backend and frontend.

Backend tests

• Unit: repository/service/storage adapter (mock MinIO), validators.
• Integration: API with test DB + MinIO container, stream upload/download, auth/tenant checks.

Frontend tests

• Unit: components/forms, upload interactions, previews.
• Integration: hooks with mocked API; navigation flows for list/detail/upload.

Commands

• make test (backend + frontend)
• make shell-backend then npm test -- features/documents
• make test-frontend

Done when

• All tests/linters pass with zero issues; upload/download E2E verified in containers.

Status

• Backend unit tests (service/repo/storage; validators): pending ☐
• Backend integration tests (upload/download/auth/tenant): pending ☐
• Frontend unit tests (components/forms/uploads/previews): pending ☐
• Frontend integration tests (hooks + navigation flows): pending ☐
• CI via make test and linters green: pending ☐

Phase 7 — Reality Checkpoints and Handoff

Checkpoints

• After each phase: make rebuild && make logs.
• Before moving on: Verify auth + tenant pre‑handlers, ownership checks, and mobile responsiveness.
• When interrupted: Commit current status and annotate the “Current Handoff Status” section below.

Handoff fields (update as you go)

• Storage façade: [x] implemented [ ] validated against MinIO
• Multipart plugin: [x] registered [x] enforcing limits
• Documents migrations: [x] added [ ] executed [ ] indexes verified
• Repo/service/routes: [x] implemented [x] ownership checks
• Frontend routes/nav: [x] added [x] mobile [x] desktop
• Upload/download flows: backend [x] implemented UI [x] progress [x] preview [ ] signed URLs (optional)
• Tests: [ ] unit backend [ ] int backend [ ] unit frontend [ ] int frontend

Diagnostics Notes

• Added /api/health endpoint in backend to validate Traefik routing to admin-backend for API paths.
• Fixed Fastify schema boot error by removing Zod schemas from documents routes (align with existing patterns). This prevented route registration and caused 404 on /api/* while server crashed/restarted.

S3 Compatibility Notes

• The interface is provider‑agnostic. MinIO adapter speaks S3‑compatible API using custom endpoint and credentials from getMinioConfig().
• Adding AWS S3 later: Implement backend/src/core/storage/adapters/s3.adapter.ts using @aws-sdk/client-s3, wire via a simple provider flag (e.g., storage.provider: 'minio' | 's3'). No feature code changes expected.
• Security parity: Keep private objects by default; consider server‑side encryption when adding AWS S3.

Reference Pointers

• MinIO config: backend/src/core/config/config-loader.ts (getMinioConfig()) and config/app/production.yml.
• Auth plugin: backend/src/core/plugins/auth.plugin.ts.
• Tenant middleware: backend/src/core/middleware/tenant.ts.
• Migration runner: backend/src/_system/migrations/run-all.ts (edit MIGRATION_ORDER).
• Feature registration: backend/src/app.ts (register documentsRoutes and update /health).
• Frontend nav and layout: frontend/src/App.tsx, frontend/src/core/store/navigation.ts, frontend/src/shared-minimal/components/mobile/GlassCard.

Success Criteria

• Documents CRUD with upload/download works on mobile and desktop.
• Ownership and tenant enforcement on every request; private object storage; safe file types.
• S3‑compatible storage layer with MinIO adapter; S3 adapter can be added without feature changes.
• All tests and linters green; migrations idempotent and ordered after vehicles.
• Build hygiene: backend TS errors fixed (unused import, override modifier, union narrowing) ✓