Skip to main content

Design: MLCDFSupplyModel — per-VIN ML-derived supply matrix

Status: design only. Not yet implemented.

Motivation​

The existing MatrixSupplyModel builds P(vehicle on lot at day d) using population-level parametric distributions (Weibull DOL, Burr12 DIT) fit at the model level and dispatched on canonical vehicle status. This is OEM-agnostic but coarse: every vehicle of a given model shares the same distribution parameters regardless of VIN-level characteristics.

BMW has a chained ML pipeline (pit_days_until_dealerarrival → pit_days_until_sale) that produces per-VIN CDF predictions conditioned on logistics/pipeline features (priority code, stop-sale, event ID, series, market). This design describes how to use those per-VIN CDFs to build a richer supply matrix, as a second optional SupplyModel implementation.

What the BMW models provide​

pit_days_until_dealerarrival_inference (BigQuery):

  • id — VIN identifier (maps to order_id in the inventory schema)
  • arrived_within_n_days_cdf — ARRAY<STRUCT<n_days INT64, prob_arrived_within_n_days FLOAT64>>, 201 elements covering days 0–200. Entry i is P(vehicle arrives at dealer by day i from snapshot date). Linear interpolation within each of the 13 training buckets is applied in the inference table itself.
  • prob_after_200 — residual probability mass beyond day 200.
  • Grain: per-(id, date) — one row per VIN per daily snapshot.

pit_days_until_sale_inference (BigQuery):

  • id
  • sold_within_n_days_cdf — ARRAY<STRUCT<n_days INT64, prob_sold_within_n_days FLOAT64>>, 201 elements covering days 0–200. Linear interpolation within each of the 5 training buckets is applied in the inference table itself, so consumers receive a per-day CDF with no further interpolation needed. Structure is identical to arrived_within_n_days_cdf.
  • prob_after_200 — residual probability mass beyond day 200.
  • Grain: latest-per-VIN (one row per VIN, already filtered to NOT sold).

P(on lot at day d) derivation per vehicle status​

INV (on lot now)​

P(on lot at day d) = P(not sold by day d) = 1 - sale_CDF(d)

Directly replaces _prob_inv's Weibull conditional survival. No additional convolution needed — the sale model's survival function is the on-lot curve.

IP / OTW / VPC (pipeline vehicles)​

P(on lot at day d) ≈ arrival_CDF(d) − sale_CDF(d) (clipped to [0, 1])

arrival_CDF(d) is the per-VIN probability the vehicle has arrived at the dealer by day d. sale_CDF(d) is the per-VIN probability the vehicle has sold by day d. Their difference estimates the probability the vehicle is present on the lot — arrived but not yet sold. Clipping handles the narrow numerical region where the sale probability rises faster than the arrival probability (can occur near day 0 for vehicles very close to dealer arrival).

The sale model already receives approx_ev_days_until_dealerarrival as a feature, making it implicitly aware of pipeline state: the CDF effectively conditions on when the vehicle is expected to arrive without being explicitly conditioned on an exact arrival date. This makes the arrival × DOL convolution unnecessary — the sale model absorbs the pipeline lag in its own CDF. No Weibull DOL parameters are needed for this path.

Architecture​

interfaces/supply_model.py  (abstract base — unchanged)
    ├── supply_models/matrix.py       MatrixSupplyModel   type: "matrix"
    │       Weibull/Burr12 parametric, status-dispatch, OEM-agnostic
    │
    └── supply_models/ml_cdf.py       MLCDFSupplyModel    type: "ml_cdf"
            Per-VIN CDFs from inference tables, BMW-specific inputs

MLCDFSupplyModel implements the same SupplyModel abstract interface:

  • compute(context) -> SupplyMatrix
  • update_from_candidate(candidate, order_id, production_date) -> SupplyMatrix
  • compute_on_lot_curve(model_id, production_date, scope_date) -> np.ndarray

Switching between paths is a one-line YAML change:

# Parametric path (Mercedes, or BMW without inference tables)
supply_model:
  type: matrix

# ML CDF path (BMW with inference tables available)
supply_model:
  type: ml_cdf

New data loaders required​

Both loaders live in data_loaders/bmw.py alongside the existing BMW loaders.

BMWSaleCDFLoader type: "bmw_sale_cdf"​

Reads pit_days_until_sale_inference. The CDF column is sold_within_n_days_cdf: ARRAY<STRUCT<n_days INT64, prob_sold_within_n_days FLOAT64>>, so unpacking requires accessing the .prob_sold_within_n_days struct field per element, not just indexing a plain numeric array. Output: wide DataFrame keyed by order_id with columns day_0 through day_200.

BMWArrivalCDFLoader type: "bmw_arrival_cdf"​

Reads pit_days_until_dealerarrival_inference. Column is arrived_within_n_days_cdf: ARRAY<STRUCT<n_days INT64, prob_arrived_within_n_days FLOAT64>>, same shape. Grain is per-(id, date) — unlike the sale table, one row exists per VIN per snapshot day. The loader must filter to today's snapshot before unpacking, e.g.:

WHERE date = CURRENT_DATE()

or equivalently QUALIFY ROW_NUMBER() OVER (PARTITION BY id ORDER BY date DESC) = 1. Output: wide DataFrame keyed by order_id with columns day_0 through day_200.

BigQuery ARRAY<STRUCT> → wide-column pivot options:

  • UNNEST + pivot in the query (verbose SQL, but keeps data transfer minimal)
  • Read the ARRAY column into Polars as a List-of-Struct column, then .explode() + access struct fields + pivot in Python (simpler query, more client-side work)

New datasets in bmw_ml_cdf.yaml​

  - dataset: vehicle_sale_cdf
    type: bmw_sale_cdf
    project: "bmw-prod-309117"
    table: "bmw-prod-309117.predictive_ordering.pit_days_until_sale_inference"

  - dataset: vehicle_arrival_cdf
    type: bmw_arrival_cdf
    project: "bmw-prod-309117"
    table: "bmw-prod-309117.predictive_ordering.pit_days_until_dealerarrival_inference"

MLCDFSupplyModel.compute() outline​

1. inventory = context.dealer_inventory          # canonical DealerInventory schema
2. sale_cdf  = context.get("vehicle_sale_cdf")   # wide DataFrame: order_id + day_0..day_200
3. arr_cdf   = context.get("vehicle_arrival_cdf")# wide DataFrame: order_id + day_0..day_200
4. dol       = context.model_dol_parameters      # Weibull params, fallback only
5. For each vehicle:
     - join sale_cdf and arr_cdf on order_id
     - status == ON_LOT:   on_lot_curve = 1 − sale_cdf row  (survival function)
     - status == pipeline: on_lot_curve = clip(arr_cdf row − sale_cdf row, 0, 1)
     - no CDF available:   fall back to parametric (Weibull/Burr12) same as MatrixSupplyModel
6. Assemble SupplyMatrix (same shape as today: order_id rows × day_0..day_364 columns)

Step 5's fallback means MLCDFSupplyModel degrades gracefully when inference data is missing for some VINs (e.g. new arrivals not yet scored, or VINs outside the model's training domain).

update_from_candidate() behaviour​

Newly selected templates have no pre-computed per-VIN CDF (inference tables are batch, not real-time). For these, fall back to compute_on_lot_curve() using population-level DOL parameters — identical to MatrixSupplyModel.update_from_candidate() today. This is acceptable: the selected template is one row among many, and the parametric fallback for a single added vehicle does not materially change the supply take-rate signal.

compute_on_lot_curve() dispatches on model_id. A candidate template carries series_id / sports_group but may not carry a model_id in the same form used as the DOL parameter table key. Verify that the candidate schema includes model_id or that a lookup from (series_code, series_name, sports_group) → model_id is available before wiring up this fallback.

Open questions​

  1. Sale CDF structure: resolved. pit_days_until_sale_inference emits sold_within_n_days_cdf: ARRAY<STRUCT<n_days INT64, prob_sold_within_n_days FLOAT64>> over days 0–200, with linear interpolation already applied within each training bucket. No coarser CDF on the consumer side; structure is identical to the arrival table.

  2. Pipeline vehicle simplification: resolved. sold_within_n_days_cdf fully replaces the Weibull/DOL convolution for pipeline vehicles. P(on lot at d) ≈ arrival_CDF(d) − sale_CDF(d), clipped to [0, 1]. The sale model receives approx_ev_days_until_dealerarrival as a feature, so it is implicitly aware of pipeline state and absorbs the arrival lag without requiring an explicit convolution over arrival day. No Weibull DOL parameters are needed in the ML CDF path.

  3. Empty supply rates when total ≈ 0: both MatrixSupplyModel and MLCDFSupplyModel return all-zero supply rates when temporal overlap is near zero (all vehicles' on-lot curves have decayed by the candidate's expected arrival horizon). The DemandSupplyGapScorer should be verified to handle a zero-supply dict without producing NaN or divide-by-zero scores — this is a gap in the current implementation regardless of which supply model is used.

  4. BMW status mapping: BMWMatrixInventoryLoader currently reads order_status raw from BigQuery without applying a canonical PipelineStatus mapping. Verify that the BMW inventory table already stores canonical values, or add a mapping analogous to _MERCEDES_STATUS_MAP in data_loaders/mercedes.py before the ML path can be used.