motovaultpro/.claude/skills/mvp-validation-and-qa/SKILL.md

name, description

name

description

mvp-validation-and-qa

Load before claiming any MotoVaultPro change is "done", "tested", or "ready to merge"; before running or adding tests; when wondering whether CI ran the tests (it did not); when integration tests could wipe the dev database (DROP TABLE CASCADE); or when you need the mobile+desktop verification procedure (320/768/1920px on staging). Defines the evidence bar and definition of done, maps the real test suites, and gives exact commands to run and add tests. Jest failure mechanics ("jest did not exit", tdd-guard-jest reporter errors, "npm test fails at repo root") are homed in mvp-build-and-env; raw symptom triage starts in mvp-debugging-playbook.

Validation and QA: what counts as evidence

When to use / When NOT to use

Use this skill when you are about to run tests, write tests, verify a change, or declare work complete on MotoVaultPro.

Do NOT use it for:

• Issue/branch/PR mechanics, labels, or the RULE 0/1/2 review taxonomy — see mvp-change-control.
• Diagnosing a live failure (logs, Grafana, container debugging) — see mvp-debugging-playbook and mvp-diagnostics-and-logging.
• Deploy/rollback procedure or staging environment operations — see mvp-run-and-operate.
• Setting up a machine or understanding why local dev is limited — see mvp-build-and-env.
• Making CI actually enforce any of this — that is the campaign in mvp-deploy-safety-campaign.

1. The evidence bar (read this first)

CI gates nothing beyond build + boot. Verified 2026-07-07 against .gitea/workflows/staging.yaml: the only PR-triggered workflow builds 3 Docker images (backend, frontend, ocr — TypeScript compiles because npm run build runs inside the Dockerfiles), deploys them over the shared staging environment, and asserts 5 containers pass health checks and https://staging.motovaultpro.com/api/health reports healthy with 13 required features. Zero tests, zero lint, zero type-check-as-gate, zero security scan, zero viewport check. A "green" PR proves only that the code compiles and the app boots.

Consequence: validation is entirely the author's job. If you did not run it, it did not run. Anywhere docs imply CI runs "integration tests" or "viewport validation" (root CLAUDE.md does), the code wins: it does not.

Definition of done for any change

Every box, personally executed, before requesting merge:

• npm run lint green in every touched workspace (backend/, frontend/; run from that directory — root package.json has NO scripts)
• npm run type-check green in every touched workspace
• Unit tests no worse than the known-red main baseline in every touched workspace, and green for everything your change touches (section 4 says where each suite can actually run)
• NEW tests written for new behavior — not just existing tests still passing
• Mobile AND desktop verified per the procedure in section 2 (hard project requirement; no CI substitute exists)
• Feature verified end-to-end ON STAGING via the PR deploy, before merge. Every PR push auto-deploys to https://staging.motovaultpro.com (last push wins on the shared environment). There is no fully working local dev loop, so staging IS the end-to-end environment.
• Old code deleted — replacement means removal, not accumulation
• PR template test-plan checkboxes filled in truthfully (they are the only artifact of testing; see section 6)

The human owner's RULE 0/1/2 review (see mvp-change-control) substitutes for CI today. Do not make the reviewer discover that a checkbox was aspirational.

2. Mobile + desktop verification procedure

"ALL features MUST be implemented and tested on BOTH mobile and desktop" is a hard project requirement, and there is no viewport CI of any kind. The procedure, run against staging after your PR deploys:

Widths to check (project convention, from the repo's agent definitions): 320px and 768px (mobile), 1920px (desktop). Use browser devtools responsive mode against https://staging.motovaultpro.com.

The mobile fork is a separate code path, not CSS. frontend/src/App.tsx sets mobileMode when window.innerWidth <= 768 (or a mobile user agent on resize) and then renders dedicated *MobileScreen components driven by a Zustand screen switcher — not the desktop router. So the first mobile question is existence, not layout: does your feature exist in the mobile screen switcher at all?

A new page/screen must run the CANONICAL registration checklist in mvp-architecture-contract invariant 2.7 (7 steps covering desktop route, mobile union/maps, lazy import, render block, and navigation entry point) — miss one and the feature is silently absent or unreachable on one platform. The two steps most often missed on mobile are the lazy import and the navigation entry point: a screen can be fully "registered" in the maps yet have no way to reach it.

Checklist per viewport:

• 320px: feature reachable via bottom nav or hamburger drawer; no horizontal scroll; touch targets usable; forms submit; screen switcher shows the screen.
• 768px: still mobile mode (<= 768 is mobile) — verify the boundary renders correctly.
• 1920px: desktop component path renders; feature reachable via desktop navigation; layout uses the space.
• Both paths: exercise the actual flow (create/edit/delete), not just "the page loads".

Only 3 frontend unit tests simulate viewports (by mocking matchMedia/useMediaQuery); unit tests do not discharge this requirement. Manual verification on staging does.

3. Test-suite reality map (verified 2026-07-07)

Workspace	Files	Cases	Runs where	Hazards
backend/	37 .test.ts	~500	Unit: host or container. Integration: container only	Integration tests are DESTRUCTIVE (below); jest hangs without --forceExit
frontend/	31 in src/	279	Host only, via npx jest --reporters=default (see mvp-build-and-env). No container run exists: the shipped image is nginx-only and .dockerignore excludes *.test.*	npm test broken on host (tdd-guard-jest reporter + hardcoded Linux projectRoot); known-red baseline on main (mvp-deploy-safety-campaign Phase 0.3)
ocr/	16 test_*.py	~310	Host with Python env, or container	Never run in CI, ever. All google-genai/Gemini calls are mocked — green OCR tests prove nothing about live Gemini

Facts that will surprise you:

• Integration tests are destructive. 2 of the 10 backend integration suites (vehicles, admin) run real migration SQL in beforeAll and DROP TABLE ... CASCADE in afterAll against whatever database the pool points at (e.g. backend/src/features/vehicles/tests/integration/vehicles.integration.test.ts: DROP TABLE IF EXISTS vehicles CASCADE plus a function drop). Most of the others run DELETE FROM against live tables (stations.api.test.ts:35 wipes station_cache unscoped); fuel-logs.integration.test.ts is an inert stub (no real DB work). Treat the whole set as destructive. Owner non-negotiable: no destructive database operation without a fresh backup. Run them only against a disposable database, and make db-backup first if there is any chance the pool points at data you care about. Suites can also interfere with each other (shared tables).
• Orphaned tests never run. frontend/test/fuel-logs/{FuelLogForm,useFuelGrades}.test.tsx sit outside jest roots: ['<rootDir>/src'] and are silently skipped by every run. FuelLogForm — the highest-traffic form in the app — has NO other tests. (The frontend/test/__mocks__/ files ARE used, via moduleNameMapper; only the test files are dead.) Do not add tests under frontend/test/.
• 8 of 21 backend feature capsules have zero test files (verified 2026-07-07): email-ingestion, notifications, onboarding, ownership-costs, subscriptions, terms-agreement, user-preferences, user-profile. None of these have frontend tests either. Touching them means writing the first test.
• Zero skipped tests project-wide. No .skip/.only/xit/pytest.mark.skip anywhere. Keep it that way — a skipped test here is invisible debt with no CI to surface it.
• The lone Cypress spec is unrunnable. frontend/cypress/e2e/stations.cy.ts has no cypress.config.* anywhere, no cypress dependency in any package.json, uses an undefined cy.login() command, and references an npm run e2e script that does not exist. Do NOT model new e2e work on it; there is no working e2e framework in this repo today (root @playwright/test dependency is equally unused — no config, no specs).
• Root package.json has no scripts. npm test at repo root fails. Always cd backend/, cd frontend/, or cd ocr/ first. make lint and make type-check fan out to backend+frontend on the host and work.

4. How to run each suite

Backend

Unit tests work on the host (node_modules present). Jest does NOT exit on its own — the module-level pg pool in backend/src/core/config/database.ts and the Redis client hold open handles. Always pass --forceExit locally:

cd backend
npm test -- --testPathPattern=src/features/vehicles/tests/unit --forceExit   # one feature, unit only
npm test -- --testPathPattern=src/features/stations --forceExit              # whole feature
npm run test:feature --feature=vehicles                                      # same filter via npm config (add -- --forceExit)
npm run lint && npm run type-check                                           # both work on host

Integration tests need real Postgres+Redis and container config paths (the config loader defaults to /app/config/production.yml and /run/secrets) — and they are currently runnable NOWHERE as-shipped. The shipped backend image is the production stage (npm ci --omit=dev, no jest; only dist/ plus migration SQL copied), so docs/TESTING.md's make shell-backend + npm test recipe fails with "jest: not found" — that doc is stale here. The only viable paths are a builder-stage image or a host run against an ephemeral Postgres/Redis: see mvp-deploy-safety-campaign 1B. Whatever the vehicle: make db-backup first (non-negotiable — see the destructive-tests bullet above).

Frontend

npm test is broken on the host (tdd-guard-jest reporter + hardcoded Linux projectRoot), and there is NO container alternative: the shipped frontend image is the nginx production stage (no node) and frontend/.dockerignore excludes *.test.* from every stage, so docker compose exec mvp-frontend npm test (still documented in docs/TESTING.md) cannot work. The verified host fallback — and the single home for its mechanics and traps — is mvp-build-and-env section 2:

cd frontend
npx jest --reporters=default                          # full suite (path BEFORE flags for single files)
npx jest --reporters=default --testPathPattern=src/features/stations
npm run lint && npm run type-check                    # these work normally on the host

Expect the known-red baseline on main (17 failing tests as of 2026-07-09; numbers homed in mvp-deploy-safety-campaign Phase 0.3) — diff your run against it rather than expecting green.

OCR

cd ocr
python -m pytest                    # needs a Python env with ocr/requirements.txt installed (pytest>=7.4.0 is in it)
python -m pytest tests/test_gemini_engine.py -v

No pytest.ini/pyproject.toml; tests import from app.main import app, so run from ocr/. Remember: CI never runs these — an OCR change with green local pytest has still never been machine-checked anywhere else.

Everything cheap at once (host)

make lint && make type-check        # backend + frontend lint and tsc

5. How to add tests

Conventions

What	Where
Backend unit tests	backend/src/features/{name}/tests/unit/*.test.ts
Backend integration tests	backend/src/features/{name}/tests/integration/*.test.ts
Alternate backend convention (audit-log uses it)	backend/src/features/{name}/__tests__/*.test.ts
Backend fixtures	backend/src/features/{name}/tests/fixtures/ (JSON or TS: fuel-logs.fixtures.json, stations' mock-google-response.ts) — otherwise inline mock objects; both are established
Frontend tests	frontend/src/features/{name}/__tests__/ or co-located *.test.tsx — MUST be under src/ (jest roots)
Backend core/cross-cutting	backend/src/core/** (e.g. core/config/tests/feature-tiers.test.ts)
OCR	ocr/tests/test_*.py; images generated inline with PIL, no golden files

Route tests: fastify inject pattern

Do not spin up a listener. Build the app and inject (established pattern, e.g. backend/src/features/audit-log/__tests__/audit-log.routes.test.ts):

import { FastifyInstance } from 'fastify';

let app: FastifyInstance;
beforeAll(async () => {
  const { default: buildApp } = await import('../../../app');
  app = await buildApp();
});
afterAll(async () => { await app.close(); });

it('rejects requests without a valid token', async () => {
  const response = await app.inject({ method: 'GET', url: '/api/admin/audit-logs' });
  expect(response.statusCode).toBe(401);   // JWT plugin rejects before any admin check
});

To bypass JWT auth, mock the auth plugin before importing the app (see the top of vehicles.integration.test.ts: jest.mock('.../core/plugins/auth.plugin', ...) decorating authenticate to set request.user). Caveat on the reference file: its test named "should return 403 for non-admin users" actually sends a bogus bearer token and asserts 401 — the name is misleading; the codebase has no example asserting a real 403 here. Do not copy an expect(403) from that name.

Frontend mock strategy

frontend/jest.config.ts moduleNameMapper auto-mocks the API client: any import resolving to core/api/client is replaced by frontend/src/core/api/__mocks__/client.ts. Components under test never hit the network; configure responses through that mock. CSS and image imports are mapped to frontend/test/__mocks__/{styleMock,fileMock}.js.

What every new repository test MUST cover: numeric coercion round-trip

This is the project's recurring bug class: node-postgres returns PostgreSQL NUMERIC/DECIMAL columns as strings, and every repository hand-rolls parseFloat coercion in its mapRow(). Missed coercion shipped at least three separate bugfix PRs (#241, #244 era). Any new repository method returning a numeric column needs a test proving strings-in, numbers-out. Copy-pasteable, in the established style (mocked pg.Pool, matching vehicles.repository.test.ts; verified against FuelLogsRepository as it exists today):

import { Pool } from 'pg';
import { FuelLogsRepository } from '../../data/fuel-logs.repository';

describe('FuelLogsRepository numeric coercion', () => {
  let pool: Pool;
  let repository: FuelLogsRepository;

  beforeEach(() => {
    pool = { query: jest.fn() } as any;
    repository = new FuelLogsRepository(pool);
  });

  it('coerces DECIMAL columns (returned as strings by node-postgres) to numbers', async () => {
    (pool.query as jest.Mock).mockResolvedValue({
      rows: [{
        id: 'log-1',
        user_id: 'user-123',
        vehicle_id: 'veh-1',
        date: '2026-07-01',
        odometer: 42000,
        gallons: '12.345',            // pg gives NUMERIC back as string
        price_per_gallon: '3.599',
        total_cost: '44.43',
        station: null, location: null, notes: null,
        created_at: new Date(), updated_at: new Date(),
      }],
    });

    const log = await repository.findById('log-1');

    expect(log!.gallons).toBe(12.345);        // number, not '12.345'
    expect(log!.pricePerGallon).toBe(3.599);  // and snake_case -> camelCase
    expect(log!.totalCost).toBe(44.43);
  });
});

Assert on the mapped camelCase field names — that simultaneously tests the mandatory mapRow() snake_case-to-camelCase conversion. If the method under test returns raw-ish rows (the fuel-logs "enhanced" path does, via mapEnhancedRow), still assert typeof value === 'number' on every decimal field.

6. Coverage and enforcement reality

• No coverage thresholds exist. Neither backend/jest.config.js nor frontend/jest.config.ts has coverageThreshold (verified 2026-07-07). Backend collects coverage (text/lcov/html reporters) but nothing fails on any number. Do not claim "coverage-gated" anywhere.
• The honor system is the PR template. .gitea/PULL_REQUEST_TEMPLATE.md test-plan checkboxes (Unit tests / Integration tests / Manual verification plus a Commands/steps list) are the only testing artifact a PR carries. Fill in the actual commands you ran; the reviewer has no other signal.
• Want CI to enforce any of this? That is exactly the scope of mvp-deploy-safety-campaign (test/lint gates, SHA-pinned deploys). Until it lands, the RULE 0/1/2 human review in mvp-change-control is the only gate — write PRs that make that review easy.
• Pre-launch calibration: a paying-user launch is the trajectory. Untested features (section 3's list of 8) touching billing (subscriptions) or user data are the highest-risk gap; adding first tests there outranks polishing well-tested capsules.

Provenance and maintenance

Authored 2026-07-07 from direct repo inspection. Volatile facts and how to re-verify each:

Fact (as of 2026-07-07)	Re-verify with
CI runs zero tests/lint (build+boot only)	`grep -nE "npm (test
Root package.json has no scripts	python3 -c "import json; print(json.load(open('package.json')).get('scripts'))"
Backend 37 test files / ~500 cases / 10 integration	find backend/src -name "*.test.ts" | wc -l ; find backend/src -path "*tests/integration*" -name "*.test.ts" | wc -l
Frontend 31 files / 279 cases in src; 2 orphaned in frontend/test/	find frontend/src \( -name "*.test.ts" -o -name "*.test.tsx" \) | wc -l ; ls frontend/test/fuel-logs/
OCR 16 files / ~310 tests	find ocr/tests -name "test_*.py" | wc -l ; grep -rE "^\s*def test_" ocr/tests | wc -l
8 backend capsules with zero tests	for f in backend/src/features/*/; do echo "$(basename $f): $(find $f -name '*.test.ts' | wc -l)"; done
Frontend jest broken on host (tdd-guard-jest reporter, hardcoded projectRoot)	grep -n "tdd-guard-jest|projectRoot" frontend/jest.config.ts then cd frontend && npm test
Integration tests DROP TABLE CASCADE	grep -rn "DROP TABLE" backend/src --include="*.test.ts"
No coverageThreshold anywhere	grep -rn coverageThreshold backend/jest.config.js frontend/jest.config.ts (expect no hits)
Mobile breakpoint <= 768; registration checklist (canonical: mvp-architecture-contract 2.7)	grep -n "innerWidth <= 768" frontend/src/App.tsx ; grep -n "MobileScreen\b" frontend/src/core/store/navigation.ts
Cypress spec still unrunnable	find . -name "cypress.config.*" (expect none) ; grep -rn cypress frontend/package.json package.json
Zero skipped tests	`grep -rnE ".skip(
Staging health gate: 13 features	grep -n REQUIRED_FEATURES .gitea/workflows/staging.yaml