Normalization
CandidateNormalizer converts OEM candidate data into CandidatePool and
NormalizedCandidate values. Configure it under
PipelineConfig.candidates_normalizer. Pydantic selects its class through
AnyCandidateNormalizer.
Interface
CandidateNormalizer
defines four methods:
| Method | Result |
|---|---|
normalize(raw) | One NormalizedCandidate |
denormalize(norm) | OEM output fields |
normalize_candidates(df, scope) | A CandidatePool |
normalize_inventory_candidates(df, scope) | Inventory candidates keyed by vehicle identifier |
The context stores use the two DataFrame methods during assembly. Mercedes also
supports individual-row conversion. BMW supports denormalize() and raises
NotImplementedError for normalize(). Stellantis and Audi raise
NotImplementedError for both individual-row methods.
Candidate fields and metadata
types.py defines NormalizedCandidate and
CandidateAttribute. The object identifier is candidate_id; pool DataFrames
use id. See Data Models for field definitions and the
candidate pool design
for DataFrame storage.
Normalizers preserve source attribute types and codes. Scorers match them to
statistics through (attribute_type, attribute_value).
NormalizedCandidate.metadata contains additional OEM fields. Mercedes stores
elective option codes there for denormalize(). Stellantis and Audi copy the
columns selected by metadata_cols. A consumer that requires a metadata field
must configure the loader and normalizer to supply it.
OEM behavior
| Normalizer | Candidate identifier | Inventory identifier | Source label |
|---|---|---|---|
| BMW | candidate_code_id | order_id | BMW |
| Mercedes | candidate_code_id | order_id | NETSTAR for candidates; INVENTORY for inventory |
| Stellantis | inventory_id | inventory_id | STELLANTIS |
| Audi | inventory_id | inventory_id | AUDI |
BMW and Stellantis build candidate pools from DataFrame columns. Audi inherits the Stellantis implementation. Mercedes validates individual rows and builds a pool from the resulting objects.
multi_valued_attribute_types identifies types that may have several attribute
codes. For other types, the columnar normalizers reject candidates with multiple
attribute rows. When Stellantis, Mercedes and Audi read a nested
vehicle_attributes column, they first remove duplicate rows with the same type
and code. When they read a separate candidate_attributes table, they remove no
rows. featurization/inventory has one row per (inventory_id, attribute_type, attribute_value), so a duplicate row there is an error. BMW preserves duplicate
rows for this check.
Stellantis and Audi log malformed inventory attributes and skip the affected vehicle. Candidate validation failures raise exceptions.
Adding a normalizer
- Implement the DataFrame methods in
CandidateNormalizer. Implement individual-row methods when needed, or raiseNotImplementedError. - Declare a
type: Literal["..."]field on the Pydantic class. - For columnar inputs, convert source columns to the names expected by
build_pool. For inputs that require individual-row validation, useCandidatePool.from_candidates(). - Add the class to
AnyCandidateNormalizerand setcandidates_normalizer.typein the OEM configuration.