Skip to main content

Selection Pipeline

SelectionPipeline scores candidates and returns recommendations. The factory builds its context from a PipelineConfig and SelectionScope. See the quick start for a complete example.

Configuration​

PipelineConfig defines the available settings and their defaults. Pydantic selects components by type, except for selection strategies, which use approach.

A configuration chooses one of two input sources:

  • datasets declares loaders for TakeRateContextStore.
  • iceberg_tables declares tables for IcebergContextAssembler, which builds a DerivedLayerContextStore.

When iceberg_tables is set, the factory requires an Iceberg catalog, scope.date, a candidate normalizer, and SimulationSupplyModel. See Context Layer for the input requirements.

SimulationSupplyModel requires a sales_rate. This SalesRateConfig holds the sales window and the floor on each group's daily sales rate. The shipped simulation configurations leave it out, and the caller sets it with PipelineConfig.from_yaml(path, supply_model_fields=...). ai_ordering passes the sales window and forecast floor of volume optimization's supply snapshot.

Factory​

create_pipeline(config, scope) builds one pipeline. It rejects duplicate scorer output names with ValueError.

Assembly connects the configured components in this order:

  1. PipelineConfig.from_yaml() validates the configuration and constructs its component instances.
  2. The factory builds a context. For datasets, TakeRateContextStore.from_config() asks DatasetAssembler to load the inputs. For iceberg_tables, IcebergContextAssembler loads inputs for the scope.
  3. The configured normalizer converts candidate inputs to a CandidatePool and inventory inputs to NormalizedCandidate objects. The context keeps these values with its datasets and supply model.
  4. The factory passes that context to SelectionPipeline with the configured scorers and selection strategy. The pipeline is ready to score when the caller requests recommendations.

See dataset names used during assembly for the keys that select candidate inputs and the supply-rate provider.

Batch assembly​

create_batch_pipelines(config, scopes) shares dataset loading across scopes when iceberg_tables is absent. Its optional datasets argument supplies DataFrames already loaded by the caller. Otherwise, DatasetAssembler.load_all_bulk() reads the inputs for all requested scopes in one batch.

TakeRateContextStore.batch_from_config() normalizes candidates once per model. Dealer scopes share that model's pool and keep their own candidate IDs. Model-only datasets are filtered once per model and shared. Dealer-specific inputs are filtered for each scope. Each returned pipeline has its own context.

With iceberg_tables, the batch factory calls create_pipeline() for each scope. Each context receives a separate copy of the simulation supply model. The caller can provide run_datasets to share tables loaded for the run; each context reads only its scope's rows from those tables. The other inputs are loaded separately for each scope. See tables shared across a run for the supported tables and storage format.

Both factory functions accept these keyword arguments for Iceberg configurations:

ArgumentMeaning
catalogInitialized Iceberg catalog
exclude_inventory_idsVehicle identifiers to leave out of supply until restore_inventory adds them back
run_datasetsTables loaded for the run and filtered again for each scope

Recommendations and confirmation​

get_recommendations(rec_count) scores the candidate pool once and returns up to rec_count recommendations. It does not update supply between recommendations.

Each scorer reads its inputs from the context and returns named score columns. The pipeline passes these columns to the selection strategy as scoring_dict. WeightedAverageSelection applies weights and masks, then ranks the combined scores. It creates candidate objects only for the selected rows. See Scoring and Selection for the calculations.

Call confirm_order(candidate, order_id, production_date) after an order is confirmed. This updates inventory and supply before the next scoring request.

Call restore_inventory(inventory_ids) to add excluded vehicles back to supply. The next scoring request derives supply take rates with them included.

The caller controls the iterative loop:

  1. Request one recommendation with get_recommendations(rec_count=1).
  2. Submit the candidate through the ordering service and wait for confirmation.
  3. Pass the confirmed candidate and order details to confirm_order().
  4. Request the next recommendation. Its scores reflect the updated supply.

Confirmation adds the candidate to the context's inventory and updates the supply model. TakeRateContextStore also calls the supply provider's update() method. MatrixDerivedProvider derives the next rates from the updated matrix. DirectRateProvider updates its stored attribute rates directly. Requesting several recommendations in one call does not perform this confirmation loop.

The optional exclude_candidate_ids argument prevents specified candidates from being returned. Excluded candidates still participate in scoring. The acceptance scorer uses the remaining candidate IDs when deciding which dealer restrictions to apply.

Set include_metadata=True to include score ranks and take-rate details on each recommendation. pipeline.metadata identifies the pipeline. pipeline.run_metadata records candidate counts and production restrictions.

Production dates and restrictions​

rescope(production_date, production_restrictions=None, *, dealer_restrictions=None) returns a new pipeline. It preserves the original pipeline's scope and shares its supply state and inventory.

Production restrictions remove matching candidates before scoring. Dealer restrictions are passed to the context for acceptance scoring. When renormalize_demand_on_prune is enabled, the new context rescales demand rates to the attributes present in the remaining candidates.

The new pipeline keeps the same PipelineMetadata. It receives new PipelineRunMetadata with the candidate counts and production restrictions for that scope.

Manual construction​

Call SelectionPipeline directly when the caller already has a context store and component instances. Supply these constructor arguments:

  • config and scope
  • context
  • scoring_modules
  • selection_strategy

For WeightedAverageSelection, configure acceptance under scorer_masks when it should multiply the weighted score. The included configurations show this setup. See Scoring and Selection for score weighting.