feat: Google Vision primary OCR with Auth0 WIF and monthly usage cap (#127) #128

Merged

egullickson merged 8 commits from issue-127-google-vision-primary-ocr into main 2026-02-11 01:46:21 +00:00

Conversation 0 Commits 8 Files Changed 19

+647 -109

egullickson commented 2026-02-10 02:58:35 +00:00

Owner

Copy Link

Copy Source

Summary

• Invert OCR engine priority: Google Vision is now the primary engine (higher accuracy), with PaddleOCR as fallback
• Add Redis-based monthly usage counter with configurable VISION_MONTHLY_LIMIT (default 1000) -- hard cutoff to PaddleOCR when limit is reached
• Replace service account key authentication with Auth0 Workload Identity Federation (WIF) -- no credential files needed
• Add Auth0 M2M token fetch script, Docker secret mounts, and CI/CD secret injection for the two new secrets

Fixes #127

Type

• Feature
• Bug fix
• Chore / refactor
• Docs

Files Changed (19 files, +608/-86)

Area	Files	Change
OCR engines	config.py, cloud_engine.py, hybrid_engine.py, engine_factory.py	WIF auth, monthly cap logic, cloud-primary path
Auth0 script	ocr/scripts/fetch-auth0-token.sh (new)	M2M token fetcher for GCP WIF
Dockerfile	ocr/Dockerfile	Add jq, chmod script
Secrets/CI	inject-secrets.sh, staging.yaml, production.yaml	Inject AUTH0_OCR_CLIENT_ID/SECRET
Compose	All 4 compose files	Engine config, secret mounts, WIF config
Tests	test_engine_abstraction.py	17 new/updated hybrid engine tests
Config	.gitignore, google-wif-config.json	WIF config committed (not a secret)
Docs	2 .example files (new), 1 removed	Secret documentation

Test plan

• Unit tests -- 17 hybrid engine tests covering cloud-primary path, limit enforcement, counter increment/TTL, Redis failure handling, non-cloud fallback
• Integration tests -- requires container build with Auth0 secrets configured
• Manual verification -- deploy to staging after adding Gitea repo secrets

Commands / steps:

1. Create Gitea repo secrets: AUTH0_OCR_CLIENT_ID and AUTH0_OCR_CLIENT_SECRET
2. Merge PR -- staging workflow will build and deploy automatically
3. Verify OCR processes documents using Google Vision (check container logs for Google Vision client initialized via WIF)
4. Verify Redis counter increments: docker compose exec mvp-redis redis-cli -n 1 GET "ocr:vision_requests:2026-02"

Checklist

• Acceptance criteria met (from linked issue)
• No secrets committed (WIF config contains no credentials, only references)
• Logging is appropriate (no PII)
• Docs updated (example files added for new secrets)

## Summary - Invert OCR engine priority: Google Vision is now the primary engine (higher accuracy), with PaddleOCR as fallback - Add Redis-based monthly usage counter with configurable `VISION_MONTHLY_LIMIT` (default 1000) -- hard cutoff to PaddleOCR when limit is reached - Replace service account key authentication with Auth0 Workload Identity Federation (WIF) -- no credential files needed - Add Auth0 M2M token fetch script, Docker secret mounts, and CI/CD secret injection for the two new secrets Fixes #127 ## Type - [x] Feature - [ ] Bug fix - [ ] Chore / refactor - [ ] Docs ## Files Changed (19 files, +608/-86) | Area | Files | Change | |------|-------|--------| | OCR engines | `config.py`, `cloud_engine.py`, `hybrid_engine.py`, `engine_factory.py` | WIF auth, monthly cap logic, cloud-primary path | | Auth0 script | `ocr/scripts/fetch-auth0-token.sh` (new) | M2M token fetcher for GCP WIF | | Dockerfile | `ocr/Dockerfile` | Add `jq`, chmod script | | Secrets/CI | `inject-secrets.sh`, `staging.yaml`, `production.yaml` | Inject AUTH0_OCR_CLIENT_ID/SECRET | | Compose | All 4 compose files | Engine config, secret mounts, WIF config | | Tests | `test_engine_abstraction.py` | 17 new/updated hybrid engine tests | | Config | `.gitignore`, `google-wif-config.json` | WIF config committed (not a secret) | | Docs | 2 `.example` files (new), 1 removed | Secret documentation | ## Test plan - [x] Unit tests -- 17 hybrid engine tests covering cloud-primary path, limit enforcement, counter increment/TTL, Redis failure handling, non-cloud fallback - [ ] Integration tests -- requires container build with Auth0 secrets configured - [ ] Manual verification -- deploy to staging after adding Gitea repo secrets **Commands / steps:** 1. Create Gitea repo secrets: `AUTH0_OCR_CLIENT_ID` and `AUTH0_OCR_CLIENT_SECRET` 2. Merge PR -- staging workflow will build and deploy automatically 3. Verify OCR processes documents using Google Vision (check container logs for `Google Vision client initialized via WIF`) 4. Verify Redis counter increments: `docker compose exec mvp-redis redis-cli -n 1 GET "ocr:vision_requests:2026-02"` ## Checklist - [x] Acceptance criteria met (from linked issue) - [x] No secrets committed (WIF config contains no credentials, only references) - [x] Logging is appropriate (no PII) - [x] Docs updated (example files added for new secrets)

egullickson added 5 commits 2026-02-10 02:58:36 +00:00

feat: add Vision monthly cap, WIF auth, and cloud-primary hybrid engine (refs #127) ... 4abd7d8d5b

- Add VISION_MONTHLY_LIMIT config setting (default 1000)
- Update CloudEngine to use WIF credential config via ADC
- Rewrite HybridEngine to support cloud-primary with Redis counter
- Pass monthly_limit through engine factory

Co-Authored-By: Claude Opus 4.6 <[email protected]>

feat: add Auth0 WIF token script and update Dockerfile (refs #127) ... 9209739e75

- Create fetch-auth0-token.sh for Auth0 M2M -> GCP WIF token exchange
- Add jq to Dockerfile system dependencies
- Ensure script is executable in container image

Co-Authored-By: Claude Opus 4.6 <[email protected]>

feat: add Auth0 OCR secrets to injection script and CI/CD workflows (refs #127) ... 5e4848c4e2

- Add AUTH0_OCR_CLIENT_ID and AUTH0_OCR_CLIENT_SECRET to inject-secrets.sh
- Add new secrets to staging and production workflow env blocks
- Create .example files for new secret documentation

Co-Authored-By: Claude Opus 4.6 <[email protected]>

feat: update all Docker Compose files for Vision primary with WIF auth (refs #127) ... f4a28d009f

- Switch OCR engine config to google_vision primary / paddleocr fallback
- Mount Auth0 OCR secrets and WIF config into all OCR containers
- Add WIF config to repo (not a secret, contains no credentials)
- Remove obsolete google-vision-key.json.example

Co-Authored-By: Claude Opus 4.6 <[email protected]>

test: add monthly limit, counter, and cloud-primary engine tests (refs #127) ... e6dd7492a1

- Update existing hybrid engine tests for new Redis counter behavior
- Add cloud-primary path tests (under/at limit, fallback, errors)
- Add Redis counter increment and TTL verification tests
- Add Redis failure graceful handling test
- Update cloud engine error message assertion for WIF config

Co-Authored-By: Claude Opus 4.6 <[email protected]>

egullickson added 1 commit 2026-02-11 00:34:45 +00:00

fix: copy WIF config to deploy path in CI/CD workflows (refs #127) ... a416f76c21

The google-wif-config.json was never synced to the deploy path, so the
Docker bind mount created a directory artifact instead of a file. Vision
client initialization failed on every request, silently falling back to
PaddleOCR.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

egullickson added 1 commit 2026-02-11 00:41:38 +00:00

fix: capture Auth0 error response in WIF token script (refs #127) ... 7bba28154d

The set -e + curl --fail-with-body inside $() caused the script to exit
with code 22 and empty stderr, hiding the actual Auth0 error. Switch to
writing the body to a temp file and checking HTTP status manually so the
error response is visible in logs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

egullickson added 1 commit 2026-02-11 00:44:34 +00:00

fix: use correct Auth0 US region domain in WIF token script (refs #127) ... 91dc847f56

Domain was motovaultpro.auth0.com (404) instead of
motovaultpro.us.auth0.com.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

egullickson merged commit e98b45eb3a into main 2026-02-11 01:46:21 +00:00

egullickson deleted branch issue-127-google-vision-primary-ocr 2026-02-11 01:46:21 +00:00

egullickson referenced this issue from a commit 2026-02-11 01:46:22 +00:00

Merge pull request 'feat: Google Vision primary OCR with Auth0 WIF and monthly usage cap (#127)' (#128) from issue-127-google-vision-primary-ocr into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

status

backlog

status/blocked

status

done

status

in-progress

status

ready

status

review

type

bug

type

chore

type

docs

type

feature

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

egullickson

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: egullickson/motovaultpro#128