Added Documents Feature

docs/changes/DOCUMENTS.md

@@ -0,0 +1,299 @@
+# Documents Feature Plan (S3-Compatible, Phased)
+
+This plan aligns with the current codebase: MinIO is running (`admin-minio`), object storage credentials are mounted as secrets, and `appConfig.getMinioConfig()` is available. We will implement a generic S3-compatible storage surface with a MinIO-backed adapter first, following the Docker‑first, production‑only workflow and mobile+desktop requirements.
+
+— Read me quick —
+- Storage: Start with MinIO SDK via `getMinioConfig()`. Keep the interface S3‑generic to support AWS S3 later without changing features.
+- Auth/Tenant: All endpoints use `[fastify.authenticate, tenantMiddleware]`.
+- Testing: Use Jest; run via containers with `make test`.
+- Mobile+Desktop: Follow existing Zustand nav, React Router routes, GlassCard components, and React Query offlineFirst.
+
+Handoff markers are provided at the end of each phase. If work pauses, pick up from the next “Done when” checklist.
+
+## Phase 0 — Baseline Verification
+
+Objectives
+- Confirm configuration and dependencies to avoid rework.
+
+Tasks
+- Verify MinIO configuration in `config/app/production.yml` → `minio.endpoint`, `minio.port`, `minio.bucket`.
+- Verify mounted secrets exist for MinIO (`secrets/app/minio-access-key.txt`, `secrets/app/minio-secret-key.txt`).
+- Verify backend dependency presence:
+  - Present: `minio@^7.1.3`
+  - Missing: `@fastify/multipart` (add to `backend/package.json`)
+- Rebuild and tail logs
+  - `make rebuild`
+  - `make logs`
+
+Done when
+- Containers start cleanly and backend logs show no missing module errors.
+
+Status
+- MinIO configuration verified in repo (endpoint/port/bucket present) ✓
+- MinIO secrets present in repo (mounted paths defined) ✓
+- Package check: `minio` present ✓, `@fastify/multipart` added to backend/package.json ✓
+- Rebuild/logs runtime verification: pending (perform via `make rebuild && make logs`)
+
+## Phase 1 — Storage Foundation (S3-Compatible, MinIO-Backed)
+
+Objectives
+- Create a generic storage façade used by features; implement first adapter using MinIO SDK.
+
+Design
+- Interface `StorageService` methods:
+  - `putObject(bucket, key, bodyOrStream, contentType, metadata?)`
+  - `getObjectStream(bucket, key)`
+  - `deleteObject(bucket, key)`
+  - `headObject(bucket, key)`
+  - `getSignedUrl(bucket, key, { method: 'GET'|'PUT', expiresSeconds })`
+- Key scheme: `documents/{userId}/{vehicleId}/{documentId}/{version}/{uuid}.{ext}`
+- Security: Private objects only; short‑lived signed URLs when needed.
+
+Files
+- `backend/src/core/storage/storage.service.ts` — façade and factory.
+- `backend/src/core/storage/adapters/minio.adapter.ts` — uses MinIO SDK and `appConfig.getMinioConfig()`.
+
+Tasks
+- Implement MinIO client using endpoint/port/accessKey/secretKey/bucket from `appConfig.getMinioConfig()`.
+- Ensure streaming APIs are used for uploads/downloads.
+- Implement signed URL generation for downloads with short TTL (e.g., 60–300s).
+
+Done when
+- Service can put/head/get/delete and generate signed URLs against `admin-minio` bucket from inside the backend container.
+
+Status
+- Storage facade added: `backend/src/core/storage/storage.service.ts` ✓
+- MinIO adapter implemented: `backend/src/core/storage/adapters/minio.adapter.ts` ✓
+- Runtime validation against MinIO: pending (validate post-rebuild) ☐
+
+## Phase 2 — Backend HTTP Foundation
+
+Objectives
+- Enable file uploads and wire security.
+
+Tasks
+- Add `@fastify/multipart` to `backend/package.json`.
+- In `backend/src/app.ts`, register multipart with config‑based limits:
+  - `limits.fileSize` sourced from `appConfig.config.performance.max_request_size`.
+- Confirm authentication plugin and tenant middleware are active (already implemented).
+
+Done when
+- Backend accepts multipart requests and enforces size limits without errors.
+
+Status
+- Dependency added: `@fastify/multipart` ✓
+- Registered in `backend/src/app.ts` with byte-limit parser ✓
+- Runtime verification via container: pending ☐
+
+## Phase 3 — Documents Feature Capsule (Backend)
+
+Objectives
+- Create the feature capsule with schema, repository, service, routes, and validators, following existing patterns (see vehicles and fuel‑logs).
+
+Structure (backend)
+```
+backend/src/features/documents/
+├── README.md
+├── index.ts
+├── api/
+│   ├── documents.routes.ts
+│   ├── documents.controller.ts
+│   └── documents.validation.ts
+├── domain/
+│   ├── documents.service.ts
+│   └── documents.types.ts
+├── data/
+│   └── documents.repository.ts
+├── migrations/
+│   └── 001_create_documents_table.sql
+└── tests/
+    ├── unit/
+    └── integration/
+```
+
+Database schema
+- Table `documents`:
+  - `id UUID PK`
+  - `user_id VARCHAR(255)`
+  - `vehicle_id UUID` FK → `vehicles(id)`
+  - `document_type VARCHAR(32)` CHECK IN ('insurance','registration')
+  - `title VARCHAR(200)`; `notes TEXT NULL`; `details JSONB`
+  - `storage_bucket VARCHAR(128)`; `storage_key VARCHAR(512)`
+  - `file_name VARCHAR(255)`; `content_type VARCHAR(128)`; `file_size BIGINT`; `file_hash VARCHAR(128) NULL`
+  - `issued_date DATE NULL`; `expiration_date DATE NULL`
+  - `created_at TIMESTAMP DEFAULT now()`; `updated_at TIMESTAMP DEFAULT now()` with `update_updated_at_column()` trigger
+  - `deleted_at TIMESTAMP NULL`
+- Indexes: `(user_id)`, `(vehicle_id)`, `(user_id, vehicle_id)`, `(document_type)`, `(expiration_date)`; optional GIN on `details` if needed.
+
+API endpoints
+```
+POST   /api/documents                      # Create metadata (with/without file)
+GET    /api/documents                      # List (filters: vehicleId, type, expiresBefore)
+GET    /api/documents/:id                  # Get metadata
+PUT    /api/documents/:id                  # Update metadata/details
+DELETE /api/documents/:id                  # Soft delete (and delete object)
+GET    /api/documents/vehicle/:vehicleId   # List by vehicle
+POST   /api/documents/:id/upload           # Upload/replace file (multipart)
+GET    /api/documents/:id/download         # Download (proxy stream or signed URL)
+```
+- Pre‑handlers: `[fastify.authenticate, tenantMiddleware]` for all routes.
+- Validation: Zod schemas for params/query/body in `documents.validation.ts`.
+- Ownership: Validate `vehicle_id` belongs to `user_id` using vehicles pattern (like fuel‑logs).
+
+Wire‑up
+- Register in `backend/src/app.ts`:
+  - `import { documentsRoutes } from './features/documents/api/documents.routes'`
+  - `await app.register(documentsRoutes, { prefix: '/api' })`
+- Health: Update `/health` feature list to include `documents`.
+- Migrations: Add `'features/documents'` to `MIGRATION_ORDER` in `backend/src/_system/migrations/run-all.ts` after `'features/vehicles'`.
+
+Done when
+- CRUD + upload/download endpoints are reachable and secured; migrations run in correct order; ownership enforced.
+
+Status
+- Capsule scaffolded (api/domain/data/tests/migrations/README) ✓
+- Migration added `backend/src/features/documents/migrations/001_create_documents_table.sql` ✓
+- Registered routes in `backend/src/app.ts` with `/api` prefix ✓
+- Health feature list updated to include `documents` ✓
+- Migration order updated in `backend/src/_system/migrations/run-all.ts` ✓
+- CRUD handlers for metadata implemented ✓
+- Upload endpoint implemented with multipart streaming, MIME allowlist, and storage meta update ✓
+- Download endpoint implemented with proxy streaming and inline/attachment disposition ✓
+- Ownership validation on create via vehicles check ✓
+- Runtime verification in container: pending ☐
+
+## Phase 4 — Frontend Feature (Mobile + Desktop)
+
+Objectives
+- Implement documents UI following existing navigation, layout, and data patterns.
+
+Structure (frontend)
+```
+frontend/src/features/documents/
+├── pages/
+├── components/
+├── hooks/
+└── types/
+```
+
+Navigation
+- Mobile: Add “Documents” to bottom nav (Zustand store in `frontend/src/core/store/navigation.ts`).
+- Desktop: Add routes in `frontend/src/App.tsx` for list/detail/upload.
+- Sub‑screens (mobile): list → detail → upload; wrap content with `GlassCard`.
+
+Upload UX
+- Mobile camera/gallery: `<input type="file" accept="image/*" capture="environment" />`.
+- Desktop drag‑and‑drop with progress.
+- Progress tracking: React Query mutation with progress events; optimistic updates and cache invalidation.
+- Offline: Use existing React Query `offlineFirst` config; queue uploads and retry on reconnect.
+
+Viewer
+- Inline image/PDF preview; `Content-Disposition` inline for images/PDF; gestures (pinch/zoom) for mobile images.
+
+Done when
+- Users can list, upload, view, and delete documents on both mobile and desktop with responsive UI and progress.
+
+Status
+- Add Documents to mobile bottom nav (Zustand): completed ✓
+- Add desktop routes in `App.tsx` (list/detail/upload): completed ✓
+- Scaffold pages/components/hooks structure: completed ✓
+- Hook list/detail CRUD endpoints with React Query: completed ✓
+- Implement upload with progress UI: completed ✓ (hooks with onUploadProgress; UI in mobile/detail)
+- Optimistic updates: partial (invalidate queries on success) ◐
+- Offline queuing/retry via React Query networkMode: configured via hooks ✓
+- Previews: basic image/PDF preview implemented ✓ (DocumentPreview)
+- Gesture-friendly viewer: pending ☐
+- Desktop navigation: sidebar now defaults open and includes Documents ✓
+- Build hygiene: resolved TS unused import error in frontend documents hooks ✓
+
+## Phase 5 — Security, Validation, and Policies
+
+Objectives
+- Enforce safe file handling and consistent deletion semantics.
+
+Tasks
+- MIME allowlist: `application/pdf`, `image/jpeg`, `image/png`; reject executables.
+- Upload size: Enforce via multipart limit tied to `performance.max_request_size`.
+- Deletion: Soft delete DB first; delete object after. Consider retention policy later if required.
+- Logging: Create/update/delete/upload/download events include `user_id`, `document_id`, `vehicle_id` (use existing logger).
+- Optional rate limiting for upload route (defer dependency until needed).
+
+Done when
+- Unsafe files rejected; logs record document events; deletions are consistent.
+
+Status
+- MIME allowlist enforced for uploads (PDF, JPEG, PNG) ✓
+- Upload size enforced via multipart limit (config-driven) ✓
+- Deletion semantics: DB soft-delete and best-effort storage object deletion ✓
+- Event logging for document actions: pending ☐
+
+## Phase 6 — Testing (Docker-First)
+
+Objectives
+- Achieve green tests and linting across backend and frontend.
+
+Backend tests
+- Unit: repository/service/storage adapter (mock MinIO), validators.
+- Integration: API with test DB + MinIO container, stream upload/download, auth/tenant checks.
+
+Frontend tests
+- Unit: components/forms, upload interactions, previews.
+- Integration: hooks with mocked API; navigation flows for list/detail/upload.
+
+Commands
+- `make test` (backend + frontend)
+- `make shell-backend` then `npm test -- features/documents`
+- `make test-frontend`
+
+Done when
+- All tests/linters pass with zero issues; upload/download E2E verified in containers.
+
+Status
+- Backend unit tests (service/repo/storage; validators): pending ☐
+- Backend integration tests (upload/download/auth/tenant): pending ☐
+- Frontend unit tests (components/forms/uploads/previews): pending ☐
+- Frontend integration tests (hooks + navigation flows): pending ☐
+- CI via `make test` and linters green: pending ☐
+
+## Phase 7 — Reality Checkpoints and Handoff
+
+Checkpoints
+- After each phase: `make rebuild && make logs`.
+- Before moving on: Verify auth + tenant pre‑handlers, ownership checks, and mobile responsiveness.
+- When interrupted: Commit current status and annotate the “Current Handoff Status” section below.
+
+Handoff fields (update as you go)
+- Storage façade: [x] implemented  [ ] validated against MinIO
+- Multipart plugin: [x] registered  [x] enforcing limits
+- Documents migrations: [x] added  [ ] executed  [ ] indexes verified
+- Repo/service/routes: [x] implemented  [x] ownership checks
+- Frontend routes/nav: [x] added  [x] mobile  [x] desktop
+- Upload/download flows: backend [x] implemented  UI [x] progress [x] preview [ ] signed URLs (optional)
+- Tests: [ ] unit backend  [ ] int backend  [ ] unit frontend  [ ] int frontend
+
+Diagnostics Notes
+- Added `/api/health` endpoint in backend to validate Traefik routing to admin-backend for API paths.
+- Fixed Fastify schema boot error by removing Zod schemas from documents routes (align with existing patterns). This prevented route registration and caused 404 on `/api/*` while server crashed/restarted.
+
+## S3 Compatibility Notes
+
+- The interface is provider‑agnostic. MinIO adapter speaks S3‑compatible API using custom endpoint and credentials from `getMinioConfig()`.
+- Adding AWS S3 later: Implement `backend/src/core/storage/adapters/s3.adapter.ts` using `@aws-sdk/client-s3`, wire via a simple provider flag (e.g., `storage.provider: 'minio' | 's3'`). No feature code changes expected.
+- Security parity: Keep private objects by default; consider server‑side encryption when adding AWS S3.
+
+## Reference Pointers
+
+- MinIO config: `backend/src/core/config/config-loader.ts` (`getMinioConfig()`) and `config/app/production.yml`.
+- Auth plugin: `backend/src/core/plugins/auth.plugin.ts`.
+- Tenant middleware: `backend/src/core/middleware/tenant.ts`.
+- Migration runner: `backend/src/_system/migrations/run-all.ts` (edit `MIGRATION_ORDER`).
+- Feature registration: `backend/src/app.ts` (register `documentsRoutes` and update `/health`).
+- Frontend nav and layout: `frontend/src/App.tsx`, `frontend/src/core/store/navigation.ts`, `frontend/src/shared-minimal/components/mobile/GlassCard`.
+
+## Success Criteria
+
+- Documents CRUD with upload/download works on mobile and desktop.
+- Ownership and tenant enforcement on every request; private object storage; safe file types.
+- S3‑compatible storage layer with MinIO adapter; S3 adapter can be added without feature changes.
+- All tests and linters green; migrations idempotent and ordered after vehicles.
+- Build hygiene: backend TS errors fixed (unused import, override modifier, union narrowing) ✓