Files

Eric Gullickson 8376aee7ed Updates to database and API for dropdowns.

2025-11-11 10:29:02 -06:00

24 KiB

Raw Blame History

Backend Platform Repository Update - Agent 2

Task: Update VehicleDataRepository to query new database schema

Status: Ready for Implementation Dependencies: Agent 1 (Database Migration) must be complete Estimated Time: 1-2 hours Assigned To: Agent 2 (Platform Repository)

Overview

Replace normalized JOIN queries with queries against the new denormalized vehicle_options table. Change return types from {id, name}[] objects to string[] arrays.

Prerequisites

Verify Database is Ready

# Confirm Agent 1 completed successfully
docker exec mvp-postgres psql -U postgres -d motovaultpro \
  -c "SELECT COUNT(*) FROM vehicle_options;"
# Should return: 1122644

Files to Modify

backend/src/features/platform/data/vehicle-data.repository.ts
backend/src/features/platform/models/responses.ts (type definitions)

Current Implementation Analysis

File: backend/src/features/platform/data/vehicle-data.repository.ts

Current Methods (ID-based, normalized queries):

getYears(pool: Pool): Promise<number[]>
getMakes(pool: Pool, year: number): Promise<MakeItem[]>
getModels(pool: Pool, year: number, makeId: number): Promise<ModelItem[]>
getTrims(pool: Pool, year: number, modelId: number): Promise<TrimItem[]>
getEngines(pool: Pool, year: number, modelId: number, trimId: number): Promise<EngineItem[]>
decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null>

Current Type Definitions (models/responses.ts):

interface MakeItem { id: number; name: string; }
interface ModelItem { id: number; name: string; }
interface TrimItem { id: number; name: string; }
interface EngineItem { id: number; name: string; }

Step 1: Update Type Definitions

File: backend/src/features/platform/models/responses.ts

Remove Old Interfaces

Find and remove these interfaces:

export interface MakeItem {
  id: number;
  name: string;
}

export interface ModelItem {
  id: number;
  name: string;
}

export interface TrimItem {
  id: number;
  name: string;
}

export interface EngineItem {
  id: number;
  name: string;
}

Reason: API will return string[] directly, no need for wrapper objects

Note: Keep VINDecodeResult interface - VIN decode may need separate handling

Step 2: Update Repository Imports

File: backend/src/features/platform/data/vehicle-data.repository.ts

Current imports (line 6):

import { MakeItem, ModelItem, TrimItem, EngineItem } from '../models/responses';

New imports:

import { VINDecodeResult } from '../models/responses';
// MakeItem, ModelItem, TrimItem, EngineItem removed - using string[] now

Step 3: Update getYears() Method

Current Implementation (lines 14-28):

async getYears(pool: Pool): Promise<number[]> {
  const query = `
    SELECT DISTINCT year
    FROM vehicles.model_year
    ORDER BY year DESC
  `;

  try {
    const result = await pool.query(query);
    return result.rows.map(row => row.year);
  } catch (error) {
    logger.error('Repository error: getYears', { error });
    throw new Error('Failed to retrieve years from database');
  }
}

New Implementation:

async getYears(pool: Pool): Promise<number[]> {
  const query = `
    SELECT DISTINCT year
    FROM vehicle_options
    ORDER BY year DESC
  `;

  try {
    const result = await pool.query(query);
    return result.rows.map(row => row.year);
  } catch (error) {
    logger.error('Repository error: getYears', { error });
    throw new Error('Failed to retrieve years from database');
  }
}

Changes:

Line 16: FROM vehicles.model_year → FROM vehicle_options
Return type unchanged (still number[])

Step 4: Update getMakes() Method

Current Implementation (lines 33-52):

async getMakes(pool: Pool, year: number): Promise<MakeItem[]> {
  const query = `
    SELECT DISTINCT ma.id, ma.name
    FROM vehicles.make ma
    JOIN vehicles.model mo ON mo.make_id = ma.id
    JOIN vehicles.model_year my ON my.model_id = mo.id AND my.year = $1
    ORDER BY ma.name
  `;

  try {
    const result = await pool.query(query, [year]);
    return result.rows.map(row => ({
      id: row.id,
      name: row.name
    }));
  } catch (error) {
    logger.error('Repository error: getMakes', { error, year });
    throw new Error(`Failed to retrieve makes for year ${year}`);
  }
}

New Implementation:

async getMakes(pool: Pool, year: number): Promise<string[]> {
  // Use database function for optimal performance
  const query = `
    SELECT make FROM get_makes_for_year($1)
  `;

  try {
    const result = await pool.query(query, [year]);
    return result.rows.map(row => row.make);
  } catch (error) {
    logger.error('Repository error: getMakes', { error, year });
    throw new Error(`Failed to retrieve makes for year ${year}`);
  }
}

Changes:

Return type: Promise<MakeItem[]> → Promise<string[]>
Query: Use get_makes_for_year() database function
Result mapping: Return string directly (not {id, name} object)

Alternative (if not using database function):

const query = `
  SELECT DISTINCT make
  FROM vehicle_options
  WHERE year = $1
  ORDER BY make
`;

Step 5: Update getModels() Method

Current Implementation (lines 57-76):

async getModels(pool: Pool, year: number, makeId: number): Promise<ModelItem[]> {
  const query = `
    SELECT DISTINCT mo.id, mo.name
    FROM vehicles.model mo
    JOIN vehicles.model_year my ON my.model_id = mo.id AND my.year = $1
    WHERE mo.make_id = $2
    ORDER BY mo.name
  `;

  try {
    const result = await pool.query(query, [year, makeId]);
    return result.rows.map(row => ({
      id: row.id,
      name: row.name
    }));
  } catch (error) {
    logger.error('Repository error: getModels', { error, year, makeId });
    throw new Error(`Failed to retrieve models for year ${year}, make ${makeId}`);
  }
}

New Implementation:

async getModels(pool: Pool, year: number, make: string): Promise<string[]> {
  // Use database function for optimal performance
  const query = `
    SELECT model FROM get_models_for_year_make($1, $2)
  `;

  try {
    const result = await pool.query(query, [year, make]);
    return result.rows.map(row => row.model);
  } catch (error) {
    logger.error('Repository error: getModels', { error, year, make });
    throw new Error(`Failed to retrieve models for year ${year}, make ${make}`);
  }
}

Changes:

Parameter: makeId: number → make: string
Return type: Promise<ModelItem[]> → Promise<string[]>
Query: Use get_models_for_year_make() database function
Result mapping: Return string directly

Alternative (if not using database function):

const query = `
  SELECT DISTINCT model
  FROM vehicle_options
  WHERE year = $1 AND make = $2
  ORDER BY model
`;

Step 6: Update getTrims() Method

Current Implementation (lines 81-100):

async getTrims(pool: Pool, year: number, modelId: number): Promise<TrimItem[]> {
  const query = `
    SELECT t.id, t.name
    FROM vehicles.trim t
    JOIN vehicles.model_year my ON my.id = t.model_year_id
    WHERE my.year = $1 AND my.model_id = $2
    ORDER BY t.name
  `;

  try {
    const result = await pool.query(query, [year, modelId]);
    return result.rows.map(row => ({
      id: row.id,
      name: row.name
    }));
  } catch (error) {
    logger.error('Repository error: getTrims', { error, year, modelId });
    throw new Error(`Failed to retrieve trims for year ${year}, model ${modelId}`);
  }
}

New Implementation:

async getTrims(pool: Pool, year: number, make: string, model: string): Promise<string[]> {
  // Use database function for optimal performance
  const query = `
    SELECT trim_name FROM get_trims_for_year_make_model($1, $2, $3)
  `;

  try {
    const result = await pool.query(query, [year, make, model]);
    return result.rows.map(row => row.trim_name);
  } catch (error) {
    logger.error('Repository error: getTrims', { error, year, make, model });
    throw new Error(`Failed to retrieve trims for year ${year}, make ${make}, model ${model}`);
  }
}

Changes:

Parameters: Added make: string, changed modelId: number → model: string
Return type: Promise<TrimItem[]> → Promise<string[]>
Query: Use get_trims_for_year_make_model() database function
Result mapping: Return string directly

Alternative (if not using database function):

const query = `
  SELECT DISTINCT trim
  FROM vehicle_options
  WHERE year = $1 AND make = $2 AND model = $3
  ORDER BY trim
`;

Step 7: Update getEngines() Method

Current Implementation (lines 105-128):

async getEngines(pool: Pool, year: number, modelId: number, trimId: number): Promise<EngineItem[]> {
  const query = `
    SELECT DISTINCT e.id, e.name
    FROM vehicles.engine e
    JOIN vehicles.trim_engine te ON te.engine_id = e.id
    JOIN vehicles.trim t ON t.id = te.trim_id
    JOIN vehicles.model_year my ON my.id = t.model_year_id
    WHERE my.year = $1
      AND my.model_id = $2
      AND t.id = $3
    ORDER BY e.name
  `;

  try {
    const result = await pool.query(query, [year, modelId, trimId]);
    return result.rows.map(row => ({
      id: row.id,
      name: row.name
    }));
  } catch (error) {
    logger.error('Repository error: getEngines', { error, year, modelId, trimId });
    throw new Error(`Failed to retrieve engines for year ${year}, model ${modelId}, trim ${trimId}`);
  }
}

New Implementation:

async getEngines(pool: Pool, year: number, make: string, model: string, trim: string): Promise<string[]> {
  // Query vehicle_options and join with engines table
  // Handle NULL engine_id for electric vehicles
  const query = `
    SELECT DISTINCT
      CASE
        WHEN vo.engine_id IS NULL THEN 'N/A (Electric)'
        ELSE e.name
      END as engine_name
    FROM vehicle_options vo
    LEFT JOIN engines e ON e.id = vo.engine_id
    WHERE vo.year = $1
      AND vo.make = $2
      AND vo.model = $3
      AND vo.trim = $4
    ORDER BY engine_name
  `;

  try {
    const result = await pool.query(query, [year, make, model, trim]);
    return result.rows.map(row => row.engine_name);
  } catch (error) {
    logger.error('Repository error: getEngines', { error, year, make, model, trim });
    throw new Error(`Failed to retrieve engines for year ${year}, make ${make}, model ${model}, trim ${trim}`);
  }
}

Changes:

Parameters: Changed from IDs to strings (make, model, trim)
Return type: Promise<EngineItem[]> → Promise<string[]>
Query: Direct query on vehicle_options with LEFT JOIN to engines
NULL Handling: Returns 'N/A (Electric)' for NULL engine_id
Result mapping: Return string directly

Alternative using database function:

const query = `
  SELECT
    CASE
      WHEN engine_name IS NULL THEN 'N/A (Electric)'
      ELSE engine_name
    END as engine_name
  FROM get_options_for_vehicle($1, $2, $3, $4)
`;

Step 8: Add getTransmissions() Method (NEW)

Add this new method after getEngines():

/**
 * Get transmissions for a specific year, make, and model
 * Note: Transmissions are tied to model, not trim
 */
async getTransmissions(pool: Pool, year: number, make: string, model: string): Promise<string[]> {
  // Query vehicle_options and join with transmissions table
  // Handle NULL transmission_id for some vehicles
  const query = `
    SELECT DISTINCT
      CASE
        WHEN vo.transmission_id IS NULL THEN 'N/A'
        ELSE t.type
      END as transmission_type
    FROM vehicle_options vo
    LEFT JOIN transmissions t ON t.id = vo.transmission_id
    WHERE vo.year = $1
      AND vo.make = $2
      AND vo.model = $3
    ORDER BY transmission_type
  `;

  try {
    const result = await pool.query(query, [year, make, model]);
    return result.rows.map(row => row.transmission_type);
  } catch (error) {
    logger.error('Repository error: getTransmissions', { error, year, make, model });
    throw new Error(`Failed to retrieve transmissions for year ${year}, make ${make}, model ${model}`);
  }
}

Why This is New:

Old system returned hardcoded ["Automatic", "Manual"]
New database has 828 real transmission types
Returns actual data: "8-Speed Automatic", "6-Speed Manual", "CVT", etc.

Step 9: Handle VIN Decode (IMPORTANT)

Current Implementation (lines 133-164): Uses vehicles.f_decode_vin() function which may not exist after migration.

Decision Required: The VIN decode function depends on the old vehicles.* schema. You have two options:

Option A: Leave VIN Decode Unchanged (Recommended for now)

// Keep decodeVIN() method exactly as-is
// Agent 8 will investigate and fix if broken
async decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null> {
  // ... existing implementation unchanged
}

Add a comment:

/**
 * Decode VIN using PostgreSQL function
 * NOTE: This function may need updates after vehicles.* schema migration
 * See Agent 8 investigation results
 */
async decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null> {
  // ... existing code
}

Option B: Stub Out VIN Decode (If Definitely Broken)

async decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null> {
  // TODO: Agent 8 - Implement VIN decode with new schema
  logger.warn('VIN decode not yet implemented for new schema', { vin });
  throw new Error('VIN decode temporarily unavailable during migration');
}

Recommendation: Use Option A initially. Test after changes, then decide.

Step 10: Update Documentation Comments

Update the file header comment:

Old (lines 1-4):

/**
 * @ai-summary Vehicle data repository for hierarchical queries
 * @ai-context PostgreSQL queries against vehicles schema
 */

New:

/**
 * @ai-summary Vehicle data repository for hierarchical dropdown queries
 * @ai-context Queries denormalized vehicle_options table with string-based cascade
 * @ai-migration Updated to use new ETL-generated database (1.1M+ vehicle configs)
 */

Complete Updated File Structure

After all changes, the file should look like:

/**
 * @ai-summary Vehicle data repository for hierarchical dropdown queries
 * @ai-context Queries denormalized vehicle_options table with string-based cascade
 * @ai-migration Updated to use new ETL-generated database (1.1M+ vehicle configs)
 */
import { Pool } from 'pg';
import { VINDecodeResult } from '../models/responses';
import { logger } from '../../../core/logging/logger';

export class VehicleDataRepository {
  // Returns: number[]
  async getYears(pool: Pool): Promise<number[]> { ... }

  // Returns: string[] (not MakeItem[])
  async getMakes(pool: Pool, year: number): Promise<string[]> { ... }

  // Parameters: year, make (not makeId)
  // Returns: string[] (not ModelItem[])
  async getModels(pool: Pool, year: number, make: string): Promise<string[]> { ... }

  // Parameters: year, make, model (not year, modelId)
  // Returns: string[] (not TrimItem[])
  async getTrims(pool: Pool, year: number, make: string, model: string): Promise<string[]> { ... }

  // Parameters: year, make, model, trim (all strings except year)
  // Returns: string[] with 'N/A (Electric)' for NULL
  async getEngines(pool: Pool, year: number, make: string, model: string, trim: string): Promise<string[]> { ... }

  // NEW METHOD - Real transmission data (not hardcoded)
  // Parameters: year, make, model
  // Returns: string[] like "8-Speed Automatic", "CVT"
  async getTransmissions(pool: Pool, year: number, make: string, model: string): Promise<string[]> { ... }

  // VIN decode - may need updates (see Agent 8)
  async decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null> { ... }
}

Testing & Verification

Manual Testing

After making changes, test each method:

# Start a Node REPL in the backend container
docker exec -it mvp-backend node

# In Node REPL:
const { Pool } = require('pg');
const { VehicleDataRepository } = require('./src/features/platform/data/vehicle-data.repository');

const pool = new Pool({
  host: 'postgres',
  port: 5432,
  database: 'motovaultpro',
  user: 'postgres',
  password: process.env.POSTGRES_PASSWORD
});

const repo = new VehicleDataRepository();

// Test getYears
repo.getYears(pool).then(console.log);
// Expected: [2026, 2025, 2024, ..., 1980]

// Test getMakes
repo.getMakes(pool, 2024).then(console.log);
// Expected: ["Acura", "Audi", "BMW", "Ford", ...]

// Test getModels
repo.getModels(pool, 2024, 'Ford').then(console.log);
// Expected: ["Bronco", "Edge", "Escape", "Explorer", "F-150", ...]

// Test getTrims
repo.getTrims(pool, 2024, 'Ford', 'F-150').then(console.log);
// Expected: ["King Ranch", "Lariat", "Limited", "Platinum", "XL", "XLT", ...]

// Test getEngines (with trim)
repo.getEngines(pool, 2024, 'Ford', 'F-150', 'XLT').then(console.log);
// Expected: ["V6 2.7L Turbo", "V6 3.5L Turbo", "V8 5.0L", ...]

// Test getTransmissions
repo.getTransmissions(pool, 2024, 'Ford', 'F-150').then(console.log);
// Expected: ["10-Speed Automatic", "6-Speed Automatic", ...]

// Test with electric vehicle (should show N/A)
repo.getEngines(pool, 2024, 'Tesla', 'Model 3', 'Standard Range').then(console.log);
// Expected: ["N/A (Electric)"]

Unit Tests

Create or update unit tests:

File: backend/src/features/platform/tests/unit/vehicle-data.repository.test.ts

import { VehicleDataRepository } from '../../data/vehicle-data.repository';
import { Pool } from 'pg';

describe('VehicleDataRepository', () => {
  let repo: VehicleDataRepository;
  let mockPool: jest.Mocked<Pool>;

  beforeEach(() => {
    repo = new VehicleDataRepository();
    mockPool = {
      query: jest.fn()
    } as any;
  });

  describe('getMakes', () => {
    it('should return string array of makes', async () => {
      mockPool.query.mockResolvedValue({
        rows: [{ make: 'Ford' }, { make: 'Honda' }]
      } as any);

      const result = await repo.getMakes(mockPool, 2024);

      expect(result).toEqual(['Ford', 'Honda']);
      expect(mockPool.query).toHaveBeenCalledWith(
        expect.stringContaining('get_makes_for_year'),
        [2024]
      );
    });
  });

  describe('getModels', () => {
    it('should accept make string parameter', async () => {
      mockPool.query.mockResolvedValue({
        rows: [{ model: 'F-150' }, { model: 'Mustang' }]
      } as any);

      const result = await repo.getModels(mockPool, 2024, 'Ford');

      expect(result).toEqual(['F-150', 'Mustang']);
      expect(mockPool.query).toHaveBeenCalledWith(
        expect.any(String),
        [2024, 'Ford']
      );
    });
  });

  describe('getEngines', () => {
    it('should handle NULL engine_id as "N/A (Electric)"', async () => {
      mockPool.query.mockResolvedValue({
        rows: [{ engine_name: 'N/A (Electric)' }]
      } as any);

      const result = await repo.getEngines(mockPool, 2024, 'Tesla', 'Model 3', 'Standard Range');

      expect(result).toContain('N/A (Electric)');
    });
  });

  describe('getTransmissions', () => {
    it('should return real transmission data', async () => {
      mockPool.query.mockResolvedValue({
        rows: [
          { transmission_type: '10-Speed Automatic' },
          { transmission_type: '6-Speed Manual' }
        ]
      } as any);

      const result = await repo.getTransmissions(mockPool, 2024, 'Ford', 'Mustang');

      expect(result).toEqual(['10-Speed Automatic', '6-Speed Manual']);
    });
  });
});

Run tests:

cd backend
npm test -- vehicle-data.repository.test.ts

Completion Checklist

Before signaling completion:

All type imports updated (removed MakeItem, ModelItem, etc.)
getYears() queries vehicle_options table
getMakes() returns string[] and accepts year parameter
getModels() returns string[] and accepts year + make (string)
getTrims() returns string[] and accepts year + make + model (strings)
getEngines() returns string[] with NULL handling ('N/A (Electric)')
getTransmissions() method added (new)
VIN decode method addressed (left unchanged with note OR stubbed)
File documentation comments updated
Manual testing completed successfully
Unit tests pass (or updated to match new signatures)
No TypeScript compilation errors

Common Issues

Issue: "function get_makes_for_year does not exist"

Cause: Database migration not complete or functions not created

Solution:

# Verify functions exist
docker exec mvp-postgres psql -U postgres -d motovaultpro \
  -c "\df get_makes_for_year"

# If not found, Agent 1 needs to re-run migration

Issue: "column vo.make does not exist"

Cause: Old vehicles.* tables still present, vehicle_options not created

Solution:

# Check which tables exist
docker exec mvp-postgres psql -U postgres -d motovaultpro \
  -c "\dt"

# Should see: engines, transmissions, vehicle_options
# If not, Agent 1 needs to complete database migration

Issue: TypeScript errors about parameter types

Cause: Service layer (Agent 3) still using old signatures

Solution:

This is expected - Agent 3 will fix service layer
Ensure your repository signatures are correct (strings not numbers)
Agent 3 will update service to match your new signatures

Issue: Empty results returned

Cause: Case sensitivity or formatting mismatch

Solution:

# Check actual data format in database
docker exec mvp-postgres psql -U postgres -d motovaultpro <<EOF
SELECT DISTINCT make FROM vehicle_options LIMIT 10;
EOF

# Verify makes are in Title Case: "Ford" not "FORD" or "ford"

Handoff to Agent 3

Once complete, provide this information:

Updated Repository Contract

Methods:

getYears(pool: Pool): Promise<number[]>
getMakes(pool: Pool, year: number): Promise<string[]>
getModels(pool: Pool, year: number, make: string): Promise<string[]>
getTrims(pool: Pool, year: number, make: string, model: string): Promise<string[]>
getEngines(pool: Pool, year: number, make: string, model: string, trim: string): Promise<string[]>
getTransmissions(pool: Pool, year: number, make: string, model: string): Promise<string[]>
decodeVIN(pool: Pool, vin: string): Promise<VINDecodeResult | null>

Key Changes for Agent 3:

All dropdown methods return string[] (not objects)
Parameters use strings (make, model, trim) not IDs (makeId, modelId, trimId)
getTransmissions() is a new method (was hardcoded before)
NULL engines display as 'N/A (Electric)'

Verification Command

# Agent 3 can verify repository is ready:
cd backend && npm run build
# Should compile with no errors in vehicle-data.repository.ts

Completion Message Template

Agent 2 (Platform Repository): COMPLETE

Files Modified:
- backend/src/features/platform/data/vehicle-data.repository.ts
- backend/src/features/platform/models/responses.ts

Changes Made:
- Updated all dropdown methods to return string[] (not objects)
- Changed parameters from IDs to strings (make, model, trim)
- Query vehicle_options table using database functions
- Added NULL handling for electric vehicles ('N/A (Electric)')
- Added new getTransmissions() method with real data
- VIN decode left unchanged (pending Agent 8 investigation)

Verification:
✓ TypeScript compiles successfully
✓ Manual tests return correct data
✓ Unit tests pass
✓ Electric vehicles show 'N/A (Electric)' for engines
✓ Transmissions return real types (not hardcoded)

Agent 3 (Platform Service) can now update service layer to use new repository signatures.

Breaking Changes for Agent 3:
- All return types changed to string[]
- Parameter names changed: makeId→make, modelId→model, trimId→trim
- New method: getTransmissions()

Document Version: 1.0 Last Updated: 2025-11-10 Status: Ready for Implementation

24 KiB Raw Blame History

Backend Platform Repository Update - Agent 2

Task: Update VehicleDataRepository to query new database schema

Overview

Prerequisites

Verify Database is Ready

Files to Modify

Current Implementation Analysis

Step 1: Update Type Definitions

Remove Old Interfaces

Step 2: Update Repository Imports

Step 3: Update getYears() Method

Step 4: Update getMakes() Method

Step 5: Update getModels() Method

Step 6: Update getTrims() Method

Step 7: Update getEngines() Method

Step 8: Add getTransmissions() Method (NEW)

Step 9: Handle VIN Decode (IMPORTANT)

Option A: Leave VIN Decode Unchanged (Recommended for now)

Option B: Stub Out VIN Decode (If Definitely Broken)

Step 10: Update Documentation Comments

Complete Updated File Structure

Testing & Verification

Manual Testing

Unit Tests

Completion Checklist

Common Issues

Issue: "function get_makes_for_year does not exist"

Issue: "column vo.make does not exist"

Issue: TypeScript errors about parameter types

Issue: Empty results returned

Handoff to Agent 3

Updated Repository Contract

Verification Command

Completion Message Template

24 KiB

Raw Blame History