Skip to main content

Scoring and Selection

Scorers read candidates and scope data from ContextStore. Each returns named score columns. SelectionStrategy combines those columns and chooses candidates. The data loaders and normalizers handle source-specific representations.

ComponentRoleLocation
ScoringModuleProduces one or more score columnsinterfaces/scoring.py
SelectionStrategyCombines scores and selects candidatesinterfaces/strategy.py

ScoringModule​

Each scorer assembles its inputs and computes its scores independently. The pipeline separates its output into one DataFrame per score column and passes those DataFrames to the selection strategy.

  • output_columns() names the score columns.
  • score() returns candidate_id and all declared score columns.
  • Scores use the range [0, 1] so selection weights can combine them.

A scorer may return several columns from one calculation.

DemandSupplyGapScorer​

scorers/demand_supply_gap.py produces demand_supply_gap.

For each candidate attribute, the scorer computes:

weighted_gap = (demand_rate - supply_rate) * attribute_weight

It sums these values for each candidate and scales the sums to [0, 1] using the minimum and maximum candidate sums.

Demand comes from the demand_take_rate column. Supply comes from context.get_supply_take_rates(scope.model_id, scope.production_date). The context's supply provider determines whether dates affect those rates.

With zero_demand_penalty: true, a code whose demand rate is missing or at most zero_demand_max_rate (default 0) gets the gap -1 in place of demand_rate - supply_rate. Both rates lie between 0 and 1, so every code with demand has a gap above -1. A code the dealer does not sell therefore never outscores a code the dealer sells.

Without the penalty, a code with neither demand nor supply has a gap of 0. A best seller already in supply has a negative gap. The scorer then prefers the code the dealer does not sell.

The penalty is configured on the scorer:

scorers:
- type: demand_supply_gap
zero_demand_penalty: true
zero_demand_max_rate: 0.00005

Geographic smoothing gives a code the dealer never sold a small rate from the neighborhood and country layers, such as 0.00001. The Stellantis, Audi and Mercedes configs set zero_demand_max_rate: 0.00005, so for these OEMs the scorer counts a rate up to 0.00005 as unsold.

ConditionBehavior
An attribute has no demand rate, supply rate, or weightThat missing value is treated as zero, except a missing demand rate when zero_demand_penalty is true
Supply lookup raises RuntimeErrorSupply rates are treated as zero
Candidates have no attribute rowsEvery candidate receives zero
All candidate sums are equal, including a single candidateEvery candidate receives zero

AttributeTurnScorer​

AttributeTurnScorer uses type: attribute_turn and produces the attribute_turn score. It reads turn_diff from attribute_turn_diffs, matched by attribute_type and attribute_value. When iceberg_tables configures no attribute_turn_diffs table, the Iceberg loader reads the sales_abnormal_days_on_lot_diff column of demand_take_rates and renames it to turn_diff. Positive differences mean vehicles took longer to sell than the days-on-lot model predicted. Negative differences mean they sold sooner.

bad_turner_exponent defaults to 2.0, which squares positive differences. good_turner_exponent defaults to 1.0, which leaves negative differences linear. For example, a difference of five days contributes a penalty of 25, while a difference of minus five days contributes minus five. This gives large delays more influence than equally large improvements.

The scorer averages these penalties using attribute weights, negates the result, and scales the candidate scores to [0, 1]. When attribute_weights is absent, all attribute types have weight one. When it is present, missing weights count as zero.

Missing or NaN turn differences are excluded. A candidate with some usable attributes is scored from those attributes. A candidate with no usable data or no positive total weight receives 0.5, so missing data does not count as the worst performance. If the dataset is absent or empty, or all usable raw scores are equal, every candidate receives 0.5.

PriceDistributionScorer​

PriceDistributionScorer uses type: price_distribution. Each instance produces one score, named by the instance's name. For each group of its feature_groups, an instance compares the distribution of the group's price in the dealer's supply with the distribution in the model's recent sales.

A group of feature_groups is ALL_FEATURES or one of the featurization groups:

  • a top-level category, such as INTERIOR;
  • a facet group, such as TECHNOLOGY.

A vehicle's ALL_FEATURES price is its all_features_price measure, the summed price of its configurable features (see All features price). Its price for any other group is its entry for the group in the group_prices measure (see Group prices).

iceberg_tables.vehicle_measures names the measures tables, featurization/inventory_measures and featurization/synthetic_measures. The loader reads the scope model's rows into the vehicle_measures dataset. featurization/inventory_measures holds every inventory vehicle of the model, sold ones included. The dataset therefore holds the prices of the candidates, of the dealer's supply and of the model's sales. iceberg_tables.vehicle_measures.select_columns must include group_prices when an instance reads a group other than ALL_FEATURES.

As the loader reads the measures table, it replaces group_prices with one Float64 column for each group that any instance reads. Each column is named after its group. A vehicle with no entry for a group has null in the group's column. The scorer selects each group's prices from its column, so it does not reshape the list of every group's entries on every call to score.

sold_vehicles holds inventory_id, dealer_id, model_id and inferred_sold_date from consolidated/inventory. These columns record which of the model's vehicles sold, at which dealer and on which date. It is a run-level table, so an ordering run reads it once and splits it by model_id. When the loader loads a scope without run-level datasets, it reads sold_vehicles and the supply inventory in one scan of the inventory partition if:

  • sold_vehicles names the same table as the supply inventory;
  • each of the scope_filters of sold_vehicles is also one of the inventory's.

Neither sold_vehicles nor vehicle_measures may have dealer_id among its scope_filters.

For each group:

  • Demand places each of the model's sales in the lookback_days days before the scope date at its price for the group. Each of the dealer's own sales has weight 1. The sales at every dealer share a combined weight of national_weight. A dealer with few sales therefore takes most of its demand from the national sales.
  • Supply places each supply vehicle at its price for the group, with the weight from the supply simulation that supply take rates also use. A confirmed order is priced by the candidate it was confirmed from.
  • Gain is the decrease in squared maximum mean discrepancy between the two distributions when one new order is added at the candidate's price for the group. The kernel is Gaussian, with a standard deviation of bandwidth dollars.

A candidate's score is the sum of its gains over the groups, min-max scaled to [0, 1] across the scope's candidates. Every group's gain is a decrease in a squared discrepancy of the same kind, and the gains are summed before scaling. A group in which the candidates' prices barely change the discrepancy therefore adds little to the differences between their scores. A group priced the same on every vehicle adds the same gain to every candidate and leaves the scores unchanged. A separate instance for each group would instead scale each group's gains to [0, 1] on its own, so a group with little difference between candidates would count as much as one with a large difference.

bandwidth sets how far apart two prices can be and still count as the same price. With a larger bandwidth, supply vehicles priced a few thousand dollars away from a candidate count against it. The scorer computes the kernel sums on a price grid with a spacing of grid_step dollars, 50 by default. bandwidth must be at least five grid steps.

Cramér and Wasserstein distances were rejected. When supply is priced above demand, they rank the cheapest candidates best, because a new order at $0 raises the supply's cumulative distribution function at every price.

ConditionBehavior
No sales of the model in the lookback windowEvery candidate receives zero
A sold vehicle has no row in vehicle_measuresIt is not demand
No supply vehicle with positive weightCandidates are scored against demand alone
A candidate, supply vehicle or priced sale has no price for one of the groupsValueError
The scope has no date, or the context has no dealer_idValueError

Audi, Mercedes and Stellantis each configure two instances. price_distribution compares ALL_FEATURES with a bandwidth of $1,000. group_price_distribution compares every top-level category and both facet groups with a bandwidth of $500, because a single group's prices span a narrower range than all_features_price. The two instances have separate scorer_weights, so a config sets the weight of all_features_price separately from the weight of the group prices.

scorers:
- type: price_distribution
name: price_distribution
feature_groups: [ALL_FEATURES]
bandwidth: 1000
lookback_days: 180
national_weight: 10
- type: price_distribution
name: group_price_distribution
feature_groups: [INTERIOR, CHASSIS, TECHNOLOGY, DESIGN]
bandwidth: 500
lookback_days: 180
national_weight: 10

ContributionMarginScorer​

scorers/contribution_margin.py produces contribution_margin.

The scorer looks up each distinct attribute's contribution margin, sums margins per candidate, and scales the sums to [0, 1]. Equal sums produce zero scores. It requires the attribute_contribution_margin dataset.

AcceptanceScorer​

scorers/acceptance.py produces acceptance.

The score is one when a candidate satisfies every accepted dealer preference and zero otherwise.

vet_restrictions() applies the rules one at a time. It rejects a rule when fewer than min_clean_fraction of the selectable candidates, and fewer than min_clean_count of them, satisfy the rule and every rule accepted before it. min_clean_fraction defaults to 0.1 and min_clean_count to 500. With up to 5,000 selectable candidates, a tenth of them is at most 500, so only min_clean_fraction decides. With more than 5,000, a rule is kept if at least 500 of them satisfy it and every rule accepted before it. A min_clean_count of null leaves min_clean_fraction alone to decide.

The rules are applied in this order:

  1. Manual rules before inferred rules.
  2. Every other rule before a rule on an accessory code. An accessory code is an option code whose consolidated/features feature_type is ACCESSORY, such as a floor mat or a protection kit. A dealer can install or remove an accessory after production. If an accessory rule were applied first, it could leave too few satisfying candidates for a later paint, upholstery or trim rule to be accepted.
  3. The option-code rule whose code has the larger absolute effective price first. An option code's effective price is what it costs to get the code on a vehicle of the scope's model. A credit option, such as a content delete, has a negative price and ranks by the amount it credits. Attribute rules, and option-code rules whose code has no known price, rank with a price of zero.
  4. Lower estimated violation share first. A rule's estimated violation share is the Laplace-smoothed share of the dealer's vehicles that violate it, (violating_rate × observed_nobs + 1) / (observed_nobs + 2), where violating_rate is observed_rate for a must-not-have and 1 − observed_rate for a must-have. The share falls as violating_rate falls. For a violating_rate below 0.5, the share also falls as observed_nobs grows. A rule without observed_rate and observed_nobs, such as a manual rule, has a share of 0.5.
  5. Option-code rules before attribute rules.
  6. The rule that removes fewer selectable candidates first.
  7. By attribute_type and attribute_value, which makes the order deterministic.

observed_rate and observed_nobs hold the demand_rate and rate_nobs columns of the dealer_rules table (see dealer preferences). For example, a must-not-have on a paint code the dealer never took on 79 vehicles is applied before one on an equally priced paint code taken on 1 of 79 vehicles.

AcceptanceScorer reads the accessory codes from the context's accessory_codes dataset, which has an option_code column, and raises KeyError when the context has none. build_accessory_codes() derives them from every row of consolidated/features for the scope's model, so features.select_columns must include feature_type.

AcceptanceScorer reads the effective prices from the context's effective_prices dataset, which has an option_code and an effective_price column, and raises KeyError when the context has none. build_effective_prices() derives them from the configurable rows of consolidated/features for the scope's model, including the rows that are not on the order form. A code that the model offers outside a package takes the price closest to zero among its rows outside a package, and never a package's price. A code that the model offers only inside packages takes the price closest to zero among the packages that contain it. A package that the model offers only inside other packages takes its price from them by the same rule. Between a credit and a charge of the same amount, the code takes the credit's price.

The selection strategy can multiply other scores by this column as a mask.

SelectionStrategy​

The strategy receives DataFrames keyed by score column name. It returns at most rec_count recommendations in descending score order.

WeightedAverageSelection​

WeightedAverageSelection in strategies.py uses the weighted_average value of the approach discriminator.

FieldMeaning
scorer_weightsOutput column names and their weights. A nonempty mapping must sum to one.
scorer_masksOutput columns that multiply the combined score
score_normalizationOptional. The method that normalizes each score column before its weight applies, and its settings

The one method is robust_sigmoid, with a min_scale setting. Each score column that is not a mask is replaced by

z     = (score − median) / max(IQR / 1.349, or std when IQR = 0, min_scale)
score = 1 / (1 + exp(−z))

The median and the interquartile range (IQR) are taken over the target's candidates that every scorer scores and every mask keeps. IQR / 1.349 is the standard deviation of normal data with that IQR. Where more than half of those candidates share a score, the IQR is 0 and the standard deviation is the unit. min_scale bounds the unit from below, so score differences smaller than it map close to 0.5. The transform keeps each column's order, so a single score column selects the same candidates. With several columns, each is measured against its own dispersion around its median, so a column whose raw scores are bunched counts by its weight near the median. In the tails of the curve, where the best candidates sit, differences are compressed. When no candidate passes every mask, the scores are not normalized.

The combined score, which the recommendation reports as its score and final_score, is then the weighted sum of the normalized scores, not of the raw scores in module_scores.

The strategy applies each score column's weight, using one for an unspecified weight. It joins the weighted columns by candidate_id and adds them. It then multiplies the total by each mask column.

The strategy sorts by the combined score and selects the requested number of candidates. Each recommendation includes the original column scores in module_scores, as the scorers returned them. An empty scoring_dict returns no recommendations.

Dates used for supply weighting​

scope.date is day zero for inventory probability curves. scope.production_date is the expected production date of the candidate order.

DemandSupplyGapScorer passes the production date to the context's supply lookup. With MatrixDerivedProvider, the provider takes the dot product of the candidate's daily supply estimates and each inventory vehicle's daily estimates. MatrixSupplyModel estimates whether vehicles are on the lot. MLCDFSupplyModel estimates whether they have not sold, including vehicles that have not arrived. See the MLCDFSupplyModel design for that calculation.

DirectRateProvider uses supplied rates without calculating a candidate curve.

Other scoring approaches​

A different scoring approach may need its own input representation. A choice model might require a binary feature vector. A reinforcement-learning policy might require a state embedding.

Such a scorer could assemble its inputs through the existing context interface. If that preparation becomes substantial, a separate abstraction could keep it independent of the scoring calculation.

Adding a Selection Strategy​

  1. Implement SelectionStrategy as a Pydantic model with a Literal discriminator on approach.
  2. Add it to AnySelectionStrategy in strategies.py.
  3. Set selection.approach in the pipeline configuration.