Skip to main content

Candidate Coverage Checks

Two asset checks answer the question "can we recommend for this model?" over <oem>/<market>/output/candidates_output, the table that unions real inventory configurations with any synthetic ones. Both are built by build_candidate_coverage_check in ai_dagster.components.candidate_coverage_check.

checkasksseverity
models_have_candidatesdoes every orderable model carry at least one candidate?ERROR
models_meet_candidate_floordoes it carry enough?WARN

They are split because the questions differ in kind. Zero candidates is unarguable and independent of any threshold, so its red/green history stays comparable when the floor is recalibrated. Falling below a floor is a calibration judgement, and its series resets whenever the factor moves.

Why the population comes from consolidated/models

A model with no candidates writes no rows, so a group-by over the checked asset cannot see the case the checks exist to catch. The population is resolved from consolidated/models instead, and the counts are joined onto it — a model missing from the asset reads as zero.

Two settings define it, deliberately kept apart.

orderable_models_filter is a SQL predicate over that table naming what makes a model orderable — for an OEM whose ordering catalog is its own source group, the flags that catalog sets, as in code_guides_exists AND feature_catalog_exists. Each OEM's own defs.yaml records which flags it picked and why.

active_model_years says how many of the newest model years are still being allocated, counted back from the newest year the predicate matches. It defaults to 2 — usually one model year is live, and around a transition two are. Nothing bounds it at two, so it is a knob rather than an assumption.

The two are held apart because a source's model-year coverage moves on its own. A catalog backfill that widens the years a feed covers would, with the window left implicit in the predicate, widen the population with no change on our side — and one such backfill has already tripled an OEM's model table.

Prefer an over-wide predicate. A model wrongly included fails and gets investigated; a model wrongly excluded is only counted, and its gap goes unremarked.

The predicate grades, it does not exclude. Models outside it are counted in metadata — including how many of them have real inventory and still fall short — but they never drive the verdict. A predicate that is too narrow therefore cannot silence a gap; the out-of-scope count moves instead.

Left unset, nothing is in scope: the check passes and reports counts. That is the resting state for an OEM whose orderable population is not yet expressible.

The window never reaches below the partition's own year, because the asset inner-joins models on the same bound. Without that floor, a model year the asset cannot emit would be required to carry candidates it has no way to produce — which is the state from January until the next model year's catalog publishes.

The floor

floor(model) = max(min_candidates_per_model,
ceil(candidate_coverage_factor × widest choice axis))

A choice axis is an attribute type a car carries exactly one value of — one paint, one wheel diameter, one fabric. Its width is the number of distinct canonical values a model offers on it, read from attributes/features. The factor is how many candidates each of those options should carry.

Reading the floor off the catalog rather than off candidate counts keeps it clear of the generator settings that produce most of those counts: where the generator runs, most candidate rows are synthetic and bounded by max_optional_depth and max_per_base, so a floor derived from counts would move with a YAML edit to the generator. It also yields a number for an OEM on its first partition, before any operating history exists.

min_candidates_per_model is the manual lever: it raises every model's floor and takes over wherever the catalog term is small. A model with no axis left has no catalog term and sits on the absolute floor alone.

Which attribute types are axes

Every attribute type is an axis except the menus, which are named by mutually_inclusive_attribute_types — the same per-OEM declaration AttributeStatisticsComponent uses to skip types when checking that take rates sum to 1, and ProjectedDemandComponent uses to decide what to renormalize. An OEM declares it once and three components read it.

Naming the menus rather than the axes is deliberate. An undeclared type enters the maximum, and a menu is several times wider than any axis, so its floors are conspicuous; an allow-list that missed a type would silently lower floors instead, making the check permissive without saying so.

Only the widest axis enters the floor, which makes it invariant to how finely an OEM carves its taxonomy: splitting one attribute type into two cannot change a maximum, while any additive measure would double. Taxonomy granularity is exactly what differs between OEMs, which is what makes this the statistic that survives onboarding.

Left undeclared, a model's widest type is whatever the catalog offers most of.

Where the factor comes from

Past candidate counts are not a target — they were produced by heuristics nobody claims are optimal. What they establish is a lower bound: coverage that sufficed in practice should not be flagged.

The factor is calibrated across every OEM with an operating history, and set by whichever binds tightest. It is deliberately set above the largest value that leaves every past model passing, because some models that pass that bar are visibly inadequate — carrying fewer total candidates than they offer paint colors. A factor that greens those cannot detect a gap of any size.

Understand what the checks establish. Comparing a model against its peers under the same pipeline detects regression. It does not establish sufficiency — whether a count is enough to recommend well is a question about the downstream decision, and answering it needs an ablation or a backtest against realized orders.

Shortfall reasons

Every model below its floor is labeled, because a model nobody has stocked may be a launch or a hole in our inventory coverage and the two are indistinguishable from the asset:

reasonmeaning
never_stockedno rows in consolidated/inventory for this model
stocked_zeroreal cars exist and no candidate survived
thinnonzero, below the floor

Metadata

Every entry either explains a red verdict or keeps a green one honest.

keywhy it is there
below_floor_breakdownthe failing models, worst-first by how far short of their own floor they fall, each carrying that floor and its reason
models_below_floorthe size of the failure, uncapped
shortfall_reasons{reason: count} over every shortfall, so the split survives the breakdown cap
absolute_floorthe threshold behind the verdict, without which a per-model floor is unexplained
models_in_scopethe denominator: zero below floor means nothing until you know whether the check graded 150 models or none
models_with_catalog_floorreads zero when no model carries a type left after the menus are removed, which is how a break in attribute tagging surfaces rather than silently putting every model on the absolute floor
out_of_scope_stocked_below_floormodels the predicate excluded that have real inventory and still fall short — the signal that the predicate is too narrow

below_floor_breakdown is capped at MAX_BREAKDOWN_ROWS. The cap trims what is listed, never what is counted — read shortfall_reasons, not the breakdown, when totaling.

Configuration

type: ai_dagster.components.CandidatesOutputComponent
attributes:
coverage_check:
orderable_models_filter: code_guides_exists AND feature_catalog_exists
active_model_years: 2
candidate_coverage_factor: 5
mutually_inclusive_attribute_types: "{{ mutually_inclusive_attribute_types }}"
min_candidates_per_model: 1
blocking: false

Setting blocking raises the check's severity with it, since Dagster halts downstream assets only on a blocking check that failed at ERROR.

With no candidate_coverage_factor, every model sits on min_candidates_per_model, so at the default of 1 models_meet_candidate_floor and models_have_candidates ask the same question — configure a factor or a higher absolute floor for the pair to differ.

Onboarding an OEM

The predicate is per-OEM because the source-group names are. The consolidated merge emits a {source_group}_exists flag for every source group an OEM declares, and those group names are chosen per OEM.

mutually_inclusive_attribute_types is a @dg.template_var in the OEM's project, shared with its take-rate and demand components, so an OEM that has either already declares it.

To find the predicate, group the OEM's models by model_year and count each {source_group}_exists. The group whose older model years drop to zero is the ordering catalog. An OEM whose models come from a single source group has one flag that is true on every row and therefore no usable predicate; it stays unset.

Task sizing

An asset check is a step in its own right and inherits no allocation from the asset it checks, while these checks read the same partitions — the candidates table, consolidated/inventory, and the exploded feature catalog. Each check is built with the component's own op_tags, so it runs at the size the component declares rather than the agent default.