feat: Add user data import feature (Fixes #26) #27

Merged
egullickson merged 11 commits from issue-26-add-user-data-import into main 2026-01-12 03:22:32 +00:00

11 Commits

Author SHA1 Message Date
Eric Gullickson
28574b0eb4 fix: preserve vehicle identity by checking ID first in merge mode (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 2m21s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 28s
Deploy to Staging / Verify Staging (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Critical fix for merge mode vehicle matching logic.

Problem:
- Vehicles with same license plate but no VIN were matched to the same existing vehicle
- Example: 2 vehicles with license plate "TEST-123" both updated the same vehicle
- Result: "Updated: 2" but only 1 vehicle in database, second vehicle overwrites first

Root Cause:
- Matching order was: VIN → license plate
- Both vehicles had no VIN and same license plate
- Both matched the same existing vehicle by license plate

Solution:
- New matching order: ID → VIN → license plate
- Preserves vehicle identity across export/import cycles
- Vehicles exported with IDs will update the same vehicle on re-import
- New vehicles (no matching ID) will be created as new records
- Security check: Verify ID belongs to same user before matching

Benefits:
- Export-modify-import workflow now works correctly
- Vehicles maintain identity across imports
- Users can safely import data with duplicate license plates
- Prevents unintended overwrites

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 21:15:30 -06:00
Eric Gullickson
62b4dc31ab debug: add comprehensive logging to vehicle import merge (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 2m52s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 28s
Deploy to Staging / Verify Staging (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Added detailed logging to diagnose import merge issues:

- Log vehicle count at merge start
- Log each vehicle being processed with VIN/make/model/year
- Log when existing vehicles are found (by VIN or license plate)
- Log successful vehicle creation with new vehicle ID
- Log errors with full context (userId, VIN, make, model, error message)
- Log merge completion with summary statistics

This will help diagnose why vehicles show as "successfully imported" but don't appear in the UI.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 21:03:20 -06:00
Eric Gullickson
f48a18287b fix: prevent vehicle duplication and enforce tier limits in merge mode (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 2m20s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 28s
Deploy to Staging / Verify Staging (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Critical bug fixes for import merge mode:

1. Vehicle duplication bug (RULE 0 - CRITICAL):
   - Previous: Vehicles without VINs always inserted as new, creating duplicates
   - Fixed: Check by VIN first, then fallback to license plate matching
   - Impact: Prevents duplicate vehicles on repeated imports

2. Vehicle limit bypass (RULE 0 - CRITICAL):
   - Previous: Direct repo.create() bypassed tier-based vehicle limits
   - Fixed: Use VehiclesService.createVehicle() which enforces FOR UPDATE locking and tier checks
   - Impact: Free users properly limited to 1 vehicle, prevents limit violations

Changes:
- Added VehiclesService to import service constructor
- Updated mergeVehicles() to check VIN then license plate for matches
- Replace repo.create() with service.createVehicle() for limit enforcement
- Added VehicleLimitExceededError handling with clear error messages

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 20:54:38 -06:00
Eric Gullickson
566deae5af fix: match import button style to export button (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 2m39s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 37s
Deploy to Staging / Verify Staging (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Desktop changes:
- Replace ImportButton component with MUI Button matching Export style
- Use hidden file input with validation
- Dark red/maroon button with consistent styling

Mobile changes:
- Update both Import and Export buttons to use primary-500 style
- Consistent dark primary button appearance
- Maintains 44px touch target requirement

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 20:23:56 -06:00
Eric Gullickson
5648f4c3d0 fix: add import UI to desktop settings page (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 2m38s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 37s
Deploy to Staging / Verify Staging (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 6s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 20:16:42 -06:00
Eric Gullickson
197927ef31 test: add integration tests and documentation (refs #26)
All checks were successful
Deploy to Staging / Build Images (pull_request) Successful in 4m37s
Deploy to Staging / Deploy to Staging (pull_request) Successful in 29s
Deploy to Staging / Verify Staging (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Ready (pull_request) Successful in 7s
Deploy to Staging / Notify Staging Failure (pull_request) Has been skipped
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 20:05:06 -06:00
Eric Gullickson
7a5579df7b feat: add frontend import UI (refs #26)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 19:58:17 -06:00
Eric Gullickson
068db991a4 chore: Update footer 2026-01-11 19:51:34 -06:00
Eric Gullickson
a35d05f08a feat: add import service and API layer (refs #26)
Implements Milestone 3: Backend import service and API with:

Service Layer (user-import.service.ts):
- generatePreview(): extract archive, validate, detect VIN conflicts
- executeMerge(): chunk-based import (100 records/batch), UPDATE existing by VIN, INSERT new via batchInsert
- executeReplace(): transactional DELETE all user data, batchInsert all records
- Conflict detection: VIN duplicates in vehicles
- Error handling: collect errors per record, continue, report in summary
- File handling: copy vehicle images and documents from archive to storage
- Cleanup: delete temp directory in finally block

API Layer:
- POST /api/user/import: multipart upload, mode selection (merge/replace)
- POST /api/user/import/preview: preview without executing import
- Authentication: fastify.authenticate preHandler
- Content-Type validation: application/gzip or application/x-gzip
- Magic byte validation: FileType.fromBuffer verifies tar.gz
- Request validation: Zod schema for mode selection
- Response: ImportResult with success, mode, summary, warnings

Files Created:
- backend/src/features/user-import/domain/user-import.service.ts
- backend/src/features/user-import/api/user-import.controller.ts
- backend/src/features/user-import/api/user-import.routes.ts
- backend/src/features/user-import/api/user-import.validation.ts

Files Updated:
- backend/src/app.ts: register userImportRoutes with /api prefix

Quality:
- Type-check: PASS (0 errors)
- Linting: PASS (0 errors, 470 warnings - all pre-existing)
- Repository pattern: snake_case→camelCase conversion
- User-scoped: all queries filter by user_id
- Transaction boundaries: Replace mode atomic, Merge mode per-batch

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 19:50:59 -06:00
Eric Gullickson
ffadc48b4f feat: add archive extraction and validation service (refs #26)
Implement archive service to extract and validate user data import archives. Validates manifest structure, data files, and ensures archive format compatibility with export feature.

- user-import.types.ts: Type definitions for import feature
- user-import-archive.service.ts: Archive extraction and validation
- Validates manifest version (1.0.0) and required fields
- Validates all data files exist and contain valid JSON
- Temp directory pattern mirrors export (/tmp/user-import-work)
- Cleanup method for archive directories

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 19:30:43 -06:00
Eric Gullickson
e6af7ed5d5 feat: add batch insert operations to repositories (refs #26)
Add batchInsert methods to vehicles, fuel-logs, maintenance, and documents repositories. Multi-value INSERT syntax provides 10-100x performance improvement over individual operations for bulk data import.

- vehicles.repository: batchInsert for vehicles
- fuel-logs.repository: batchInsert for fuel logs
- maintenance.repository: batchInsertRecords and batchInsertSchedules
- documents.repository: batchInsert for documents
- All methods support empty array (immediate return) and optional transaction client
- Fix lint error: replace require() with ES6 import in test mock

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-11 19:28:11 -06:00