Scoring and Selection
Scorers read candidates and scope data from ContextStore. Each returns named
score columns. SelectionStrategy combines those columns and chooses candidates.
The data loaders and normalizers handle source-specific representations.
| Component | Role | Location |
|---|---|---|
ScoringModule | Produces one or more score columns | interfaces/scoring.py |
SelectionStrategy | Combines scores and selects candidates | interfaces/strategy.py |
ScoringModule
Each scorer assembles its inputs and computes its scores independently. The pipeline separates its output into one DataFrame per score column and passes those DataFrames to the selection strategy.
output_columns()names the score columns.score()returnscandidate_idand all declared score columns.- Scores use the range
[0, 1]so selection weights can combine them.
A scorer may return several columns from one calculation.
DemandSupplyGapScorer
scorers/demand_supply_gap.py produces demand_supply_gap.
For each candidate attribute, the scorer computes:
weighted_gap = (demand_rate - supply_rate) * attribute_weight
It sums these values for each candidate and scales the sums to [0, 1] using the
minimum and maximum candidate sums.
Demand comes from the demand_take_rate column. Supply comes from
context.get_supply_take_rates(scope.model_id, scope.production_date).
The context's supply provider determines whether dates affect those rates.
With zero_demand_penalty: true, a code whose demand rate is missing or at most
zero_demand_max_rate (default 0) gets the gap -1 in place of
demand_rate - supply_rate. Both rates lie between 0 and 1, so every code with demand
has a gap above -1. A code the dealer does not sell therefore never outscores a code
the dealer sells.
Without the penalty, a code with neither demand nor supply has a gap of 0. A best
seller already in supply has a negative gap. The scorer then prefers the code the dealer
does not sell.
The penalty is configured on the scorer:
scorers:
- type: demand_supply_gap
zero_demand_penalty: true
zero_demand_max_rate: 0.00005
Geographic smoothing gives a code the dealer
never sold a small rate from the neighborhood and country layers, such as 0.00001.
The Stellantis, Audi and Mercedes configs set zero_demand_max_rate: 0.00005, so for these OEMs the
scorer counts a rate up to 0.00005 as unsold.
| Condition | Behavior |
|---|---|
| An attribute has no demand rate, supply rate, or weight | That missing value is treated as zero, except a missing demand rate when zero_demand_penalty is true |
Supply lookup raises RuntimeError | Supply rates are treated as zero |
| Candidates have no attribute rows | Every candidate receives zero |
| All candidate sums are equal, including a single candidate | Every candidate receives zero |
AttributeTurnScorer
AttributeTurnScorer uses type: attribute_turn and produces the attribute_turn
score. It reads turn_diff from attribute_turn_diffs, matched by
attribute_type and attribute_value. When iceberg_tables configures no
attribute_turn_diffs table, the Iceberg loader reads the
sales_abnormal_days_on_lot_diff column of demand_take_rates and renames it to
turn_diff. Positive differences mean vehicles took longer to sell than the
days-on-lot model predicted. Negative differences mean they sold
sooner.
bad_turner_exponent defaults to 2.0, which squares positive differences.
good_turner_exponent defaults to 1.0, which leaves negative differences linear.
For example, a difference of five days contributes a penalty of 25, while a
difference of minus five days contributes minus five. This gives large delays
more influence than equally large improvements.
The scorer averages these penalties using attribute weights, negates the result,
and scales the candidate scores to [0, 1]. When attribute_weights is absent,
all attribute types have weight one. When it is present, missing weights count as
zero.
Missing or NaN turn differences are excluded. A candidate with some usable
attributes is scored from those attributes. A candidate with no usable data or
no positive total weight receives 0.5, so missing data does not count as the
worst performance. If the dataset is absent or empty, or all usable raw scores
are equal, every candidate receives 0.5.
PriceDistributionScorer
PriceDistributionScorer uses type: price_distribution. Each instance produces one
score, named by the instance's name. For each group of its feature_groups, an instance
compares the distribution of the group's price in the dealer's supply with the
distribution in the model's recent sales.
A group of feature_groups is ALL_FEATURES or one of the
featurization groups:
- a top-level category, such as
INTERIOR; - a facet group, such as
TECHNOLOGY.
A vehicle's ALL_FEATURES price is its all_features_price measure, the summed price of
its configurable features (see
All features price). Its price for any
other group is its entry for the group in the group_prices measure (see
Group prices).
iceberg_tables.vehicle_measures names the measures tables,
featurization/inventory_measures and featurization/synthetic_measures. The loader
reads the scope model's rows into the vehicle_measures dataset.
featurization/inventory_measures holds every inventory vehicle of the model, sold ones
included. The dataset therefore holds the prices of the candidates, of the dealer's supply
and of the model's sales. iceberg_tables.vehicle_measures.select_columns must include
group_prices when an instance reads a group other than ALL_FEATURES.
As the loader reads the measures table, it replaces group_prices with one Float64
column for each group that any instance reads. Each column is named after its group. A
vehicle with no entry for a group has null in the group's column. The scorer selects each
group's prices from its column, so it does not reshape the list of every group's entries
on every call to score.
sold_vehicles holds inventory_id, dealer_id, model_id and inferred_sold_date
from consolidated/inventory. These columns record which of the model's vehicles sold, at
which dealer and on which date. It is a run-level table, so an ordering run reads it once
and splits it by model_id. When the loader loads a scope without run-level datasets, it
reads sold_vehicles and the supply inventory in one scan of the inventory partition if:
sold_vehiclesnames the same table as the supply inventory;- each of the
scope_filtersofsold_vehiclesis also one of the inventory's.
Neither sold_vehicles nor vehicle_measures may have dealer_id among its
scope_filters.
For each group:
- Demand places each of the model's sales in the
lookback_daysdays before the scope date at its price for the group. Each of the dealer's own sales has weight 1. The sales at every dealer share a combined weight ofnational_weight. A dealer with few sales therefore takes most of its demand from the national sales. - Supply places each supply vehicle at its price for the group, with the weight from the supply simulation that supply take rates also use. A confirmed order is priced by the candidate it was confirmed from.
- Gain is the decrease in squared maximum mean discrepancy between the two
distributions when one new order is added at the candidate's price for the group. The
kernel is Gaussian, with a standard deviation of
bandwidthdollars.
A candidate's score is the sum of its gains over the groups, min-max scaled to [0, 1]
across the scope's candidates. Every group's gain is a decrease in a squared discrepancy of
the same kind, and the gains are summed before scaling. A group in which the candidates'
prices barely change the discrepancy therefore adds little to the differences between their
scores. A group priced the same on every vehicle adds the same gain to every candidate and
leaves the scores unchanged. A separate instance for each group would instead scale each
group's gains to [0, 1] on its own, so a group with little difference between candidates
would count as much as one with a large difference.
bandwidth sets how far apart two prices can be and still count as the same price. With a
larger bandwidth, supply vehicles priced a few thousand dollars away from a candidate count
against it. The scorer computes the kernel sums on a price grid with a spacing of
grid_step dollars, 50 by default. bandwidth must be at least five grid steps.
Cramér and Wasserstein distances were rejected. When supply is priced above demand, they rank the cheapest candidates best, because a new order at $0 raises the supply's cumulative distribution function at every price.
| Condition | Behavior |
|---|---|
| No sales of the model in the lookback window | Every candidate receives zero |
A sold vehicle has no row in vehicle_measures | It is not demand |
| No supply vehicle with positive weight | Candidates are scored against demand alone |
| A candidate, supply vehicle or priced sale has no price for one of the groups | ValueError |
The scope has no date, or the context has no dealer_id | ValueError |
Audi, Mercedes and Stellantis each configure two instances. price_distribution compares
ALL_FEATURES with a bandwidth of $1,000. group_price_distribution compares every
top-level category and both facet groups with a bandwidth of $500, because a single group's
prices span a narrower range than all_features_price. The two instances have separate
scorer_weights, so a config sets the weight of all_features_price separately from the
weight of the group prices.
scorers:
- type: price_distribution
name: price_distribution
feature_groups: [ALL_FEATURES]
bandwidth: 1000
lookback_days: 180
national_weight: 10
- type: price_distribution
name: group_price_distribution
feature_groups: [INTERIOR, CHASSIS, TECHNOLOGY, DESIGN]
bandwidth: 500
lookback_days: 180
national_weight: 10
ContributionMarginScorer
scorers/contribution_margin.py produces contribution_margin.
The scorer looks up each distinct attribute's contribution margin, sums margins
per candidate, and scales the sums to [0, 1]. Equal sums produce zero scores.
It requires the attribute_contribution_margin dataset.
AcceptanceScorer
scorers/acceptance.py produces acceptance.
The score is one when a candidate satisfies every accepted dealer preference and zero otherwise.
vet_restrictions() applies the rules one at a time. It rejects a rule when fewer
than min_clean_fraction of the selectable candidates, and fewer than
min_clean_count of them, satisfy the rule and every rule accepted before it.
min_clean_fraction defaults to 0.1 and min_clean_count to 500. With up to
5,000 selectable candidates, a tenth of them is at most 500, so only
min_clean_fraction decides. With more than 5,000, a rule is kept if at least 500
of them satisfy it and every rule accepted before it. A min_clean_count of null
leaves min_clean_fraction alone to decide.
The rules are applied in this order:
- Manual rules before inferred rules.
- Every other rule before a rule on an accessory code. An accessory code is an option
code whose
consolidated/featuresfeature_typeisACCESSORY, such as a floor mat or a protection kit. A dealer can install or remove an accessory after production. If an accessory rule were applied first, it could leave too few satisfying candidates for a later paint, upholstery or trim rule to be accepted. - The option-code rule whose code has the larger absolute effective price first. An option code's effective price is what it costs to get the code on a vehicle of the scope's model. A credit option, such as a content delete, has a negative price and ranks by the amount it credits. Attribute rules, and option-code rules whose code has no known price, rank with a price of zero.
- Lower estimated violation share first. A rule's estimated violation share is the
Laplace-smoothed share of the dealer's vehicles that violate it,
(violating_rate × observed_nobs + 1) / (observed_nobs + 2), whereviolating_rateisobserved_ratefor a must-not-have and1 − observed_ratefor a must-have. The share falls asviolating_ratefalls. For aviolating_ratebelow 0.5, the share also falls asobserved_nobsgrows. A rule withoutobserved_rateandobserved_nobs, such as a manual rule, has a share of 0.5. - Option-code rules before attribute rules.
- The rule that removes fewer selectable candidates first.
- By
attribute_typeandattribute_value, which makes the order deterministic.
observed_rate and observed_nobs hold the demand_rate and rate_nobs
columns of the dealer_rules table (see dealer preferences).
For example, a must-not-have on a paint code the dealer never took on 79 vehicles
is applied before one on an equally priced paint code taken on 1 of 79 vehicles.
AcceptanceScorer reads the accessory codes from the context's accessory_codes
dataset, which has an option_code column, and raises KeyError when the context has
none. build_accessory_codes() derives them from every row of consolidated/features
for the scope's model, so features.select_columns must include feature_type.
AcceptanceScorer reads the effective prices from the context's effective_prices
dataset, which has an option_code and an effective_price column, and raises
KeyError when the context has none. build_effective_prices() derives them from
the configurable rows of consolidated/features for the scope's model, including the
rows that are not on the order form. A code that the model offers outside a package takes the price
closest to zero among its rows outside a package, and never a package's price. A
code that the model offers only inside packages takes the price closest to zero
among the packages that contain it. A package that the model offers only inside
other packages takes its price from them by the same rule. Between a credit and a
charge of the same amount, the code takes the credit's price.
The selection strategy can multiply other scores by this column as a mask.
SelectionStrategy
The strategy receives DataFrames keyed by score column name. It returns at most
rec_count recommendations in descending score order.
WeightedAverageSelection
WeightedAverageSelection in strategies.py uses the weighted_average value of
the approach discriminator.
| Field | Meaning |
|---|---|
scorer_weights | Output column names and their weights. A nonempty mapping must sum to one. |
scorer_masks | Output columns that multiply the combined score |
score_normalization | Optional. The method that normalizes each score column before its weight applies, and its settings |
The one method is robust_sigmoid, with a min_scale setting. Each score column that is not a mask is replaced by
z = (score − median) / max(IQR / 1.349, or std when IQR = 0, min_scale)
score = 1 / (1 + exp(−z))
The median and the interquartile range (IQR) are taken over the target's candidates
that every scorer scores and every mask keeps. IQR / 1.349 is the standard deviation
of normal data with that IQR. Where more than half of those candidates share a score,
the IQR is 0 and the standard deviation is the unit. min_scale bounds the unit from
below, so score differences smaller than it map close to 0.5. The transform keeps each column's order, so a single score
column selects the same candidates. With several columns, each is measured against
its own dispersion around its median, so a column whose raw scores are bunched counts
by its weight near the median. In the tails of the curve, where the best candidates
sit, differences are compressed. When no candidate passes every mask, the scores are
not normalized.
The combined score, which the recommendation reports as its score and
final_score, is then the weighted sum of the normalized scores, not of the raw
scores in module_scores.
The strategy applies each score column's weight, using one for an unspecified
weight. It joins the weighted columns by candidate_id and adds them. It then
multiplies the total by each mask column.
The strategy sorts by the combined score and selects the requested number of
candidates. Each recommendation includes the original column scores in
module_scores, as the scorers returned them. An empty scoring_dict returns no recommendations.
Dates used for supply weighting
scope.date is day zero for inventory probability curves.
scope.production_date is the expected production date of the candidate order.
DemandSupplyGapScorer passes the production date to the context's supply lookup.
With MatrixDerivedProvider, the provider takes the dot product of the
candidate's daily supply estimates and each inventory vehicle's daily estimates.
MatrixSupplyModel estimates whether vehicles are on the lot. MLCDFSupplyModel
estimates whether they have not sold, including vehicles that have not arrived.
See the MLCDFSupplyModel design
for that calculation.
DirectRateProvider uses supplied rates without calculating a candidate curve.
Other scoring approaches
A different scoring approach may need its own input representation. A choice model might require a binary feature vector. A reinforcement-learning policy might require a state embedding.
Such a scorer could assemble its inputs through the existing context interface. If that preparation becomes substantial, a separate abstraction could keep it independent of the scoring calculation.
Adding a Selection Strategy
- Implement
SelectionStrategyas a Pydantic model with aLiteraldiscriminator onapproach. - Add it to
AnySelectionStrategyinstrategies.py. - Set
selection.approachin the pipeline configuration.