feat: Email ingestion processing service (#149) #156

New Issue

egullickson · 2026-02-13T03:52:03Z

egullickson commented

2026-02-13 03:52:03 +00:00

Relates to #149

Scope

Create the core processing service that orchestrates the email-to-record pipeline.

EmailIngestionService

Main orchestration service in domain/email-ingestion.service.ts:

Sender validation: Look up user by sender email via UserProfileRepository.getByEmail() (case-insensitive). Reject unregistered senders.
Attachment extraction: Filter attachments by supported types (PDF, PNG, JPG, JPEG, HEIC). Validate sizes (<10MB per file).
Processing coordination: For each valid attachment, delegate to receipt classifier + OCR pipeline.
Status tracking: Update email_ingestion_queue status through lifecycle (pending -> processing -> completed/failed).
Error handling: On failure, increment retry_count. If retry_count < 3, re-process. Otherwise mark as failed.
Reply emails: Send confirmation or error email back to sender.

EmailIngestionRepository

Database access layer in data/email-ingestion.repository.ts:

insertQueueEntry(), updateQueueStatus(), getQueueEntry()
insertPendingAssociation(), getPendingAssociations(), resolvePendingAssociation()
findByEmailId() for deduplication
Standard mapRow() pattern for snake_case -> camelCase

Files

backend/src/features/email-ingestion/domain/email-ingestion.service.ts
backend/src/features/email-ingestion/data/email-ingestion.repository.ts

Acceptance Criteria

Sender validation works (registered email accepted, unregistered rejected)
Attachments filtered by type and size
Queue status tracked through processing lifecycle
Retry logic works (max 3 attempts)
Error emails sent for failures

Relates to #149 ## Scope Create the core processing service that orchestrates the email-to-record pipeline. ### EmailIngestionService Main orchestration service in `domain/email-ingestion.service.ts`: 1. **Sender validation**: Look up user by sender email via `UserProfileRepository.getByEmail()` (case-insensitive). Reject unregistered senders. 2. **Attachment extraction**: Filter attachments by supported types (PDF, PNG, JPG, JPEG, HEIC). Validate sizes (<10MB per file). 3. **Processing coordination**: For each valid attachment, delegate to receipt classifier + OCR pipeline. 4. **Status tracking**: Update `email_ingestion_queue` status through lifecycle (pending -> processing -> completed/failed). 5. **Error handling**: On failure, increment retry_count. If retry_count < 3, re-process. Otherwise mark as failed. 6. **Reply emails**: Send confirmation or error email back to sender. ### EmailIngestionRepository Database access layer in `data/email-ingestion.repository.ts`: - `insertQueueEntry()`, `updateQueueStatus()`, `getQueueEntry()` - `insertPendingAssociation()`, `getPendingAssociations()`, `resolvePendingAssociation()` - `findByEmailId()` for deduplication - Standard `mapRow()` pattern for snake_case -> camelCase ### Files - `backend/src/features/email-ingestion/domain/email-ingestion.service.ts` - `backend/src/features/email-ingestion/data/email-ingestion.repository.ts` ## Acceptance Criteria - [ ] Sender validation works (registered email accepted, unregistered rejected) - [ ] Attachments filtered by type and size - [ ] Queue status tracked through processing lifecycle - [ ] Retry logic works (max 3 attempts) - [ ] Error emails sent for failures

egullickson added the

labels 2026-02-13 03:52:47 +00:00

egullickson added this to the Sprint 2026-02-02 milestone 2026-02-13 03:52:58 +00:00

egullickson referenced this issue

2026-02-13 03:54:19 +00:00

feat: Email Receipt Ingestion via Resend Webhooks #149

egullickson added

and removed

labels 2026-02-13 14:23:34 +00:00

egullickson commented

2026-02-13 14:32:26 +00:00

Milestone: Email Ingestion Processing Service

Phase: Execution | Agent: Feature Agent | Status: PASS

Completed

EmailIngestionRepository (data/email-ingestion.repository.ts): Queue CRUD operations (insertQueueEntry, updateQueueStatus, getQueueEntry, findByEmailId, getRetryableEntries), pending vehicle association management (insert, get, resolve), standard mapRow() pattern
EmailIngestionService (domain/email-ingestion.service.ts): Full processing pipeline:
- Sender validation via UserProfileRepository.getByEmail() (case-insensitive)
- Attachment extraction from webhook data or raw email fetch
- Attachment filtering by type (PDF, PNG, JPG, JPEG, HEIC) and size (<10MB)
- Dual OCR classification: tries fuel receipt + maintenance receipt, selects best result based on domain-specific fields
- Vehicle association: single-vehicle auto-associate, multi-vehicle creates pending association with in-app notification
- Status tracking through lifecycle (pending -> processing -> completed/failed)
- Retry logic (max 3 attempts with retry_count tracking)
- Templated reply emails: confirmation (receipt_processed), failure (receipt_failed), pending vehicle (receipt_pending_vehicle)
Controller updated: Replaced processEmailAsync stub with service delegation
TemplateKey extended: Added receipt_processed, receipt_failed, receipt_pending_vehicle to notifications type union
Barrel exports updated: Service, repository, and new types exported from index.ts

Quality

TypeScript type-check: PASS (zero errors)
ESLint: PASS (zero errors, only pre-existing warnings)
All 142 unit tests: PASS
Commit: e7f3728 (refs #156)

Verdict: PASS | Next: Quality review

## Milestone: Email Ingestion Processing Service **Phase**: Execution | **Agent**: Feature Agent | **Status**: PASS ### Completed - **EmailIngestionRepository** (`data/email-ingestion.repository.ts`): Queue CRUD operations (insertQueueEntry, updateQueueStatus, getQueueEntry, findByEmailId, getRetryableEntries), pending vehicle association management (insert, get, resolve), standard mapRow() pattern - **EmailIngestionService** (`domain/email-ingestion.service.ts`): Full processing pipeline: - Sender validation via UserProfileRepository.getByEmail() (case-insensitive) - Attachment extraction from webhook data or raw email fetch - Attachment filtering by type (PDF, PNG, JPG, JPEG, HEIC) and size (<10MB) - Dual OCR classification: tries fuel receipt + maintenance receipt, selects best result based on domain-specific fields - Vehicle association: single-vehicle auto-associate, multi-vehicle creates pending association with in-app notification - Status tracking through lifecycle (pending -> processing -> completed/failed) - Retry logic (max 3 attempts with retry_count tracking) - Templated reply emails: confirmation (receipt_processed), failure (receipt_failed), pending vehicle (receipt_pending_vehicle) - **Controller updated**: Replaced processEmailAsync stub with service delegation - **TemplateKey extended**: Added receipt_processed, receipt_failed, receipt_pending_vehicle to notifications type union - **Barrel exports updated**: Service, repository, and new types exported from index.ts ### Quality - TypeScript type-check: PASS (zero errors) - ESLint: PASS (zero errors, only pre-existing warnings) - All 142 unit tests: PASS - Commit: `e7f3728` (refs #156) *Verdict*: PASS | *Next*: Quality review

egullickson referenced this issue from a commit

2026-02-13 16:34:29 +00:00

feat: add email ingestion processing service and repository (refs #156)

egullickson referenced a pull request that will close this issue

2026-02-13 16:34:49 +00:00

feat: Maintenance Receipt Upload with OCR Auto-populate (#16) #161

egullickson closed this issue

2026-02-13 22:19:46 +00:00

Sign in to join this conversation.