Skip to main content

Normalization

CandidateNormalizer converts OEM candidate data into CandidatePool and NormalizedCandidate values. Configure it under PipelineConfig.candidates_normalizer. Pydantic selects its class through AnyCandidateNormalizer.

Interface​

CandidateNormalizer defines four methods:

MethodResult
normalize(raw)One NormalizedCandidate
denormalize(norm)OEM output fields
normalize_candidates(df, scope)A CandidatePool
normalize_inventory_candidates(df, scope)Inventory candidates keyed by vehicle identifier

The context stores use the two DataFrame methods during assembly. Mercedes also supports individual-row conversion. BMW supports denormalize() and raises NotImplementedError for normalize(). Stellantis and Audi raise NotImplementedError for both individual-row methods.

Candidate fields and metadata​

types.py defines NormalizedCandidate and CandidateAttribute. The object identifier is candidate_id; pool DataFrames use id. See Data Models for field definitions and the candidate pool design for DataFrame storage.

Normalizers preserve source attribute types and codes. Scorers match them to statistics through (attribute_type, attribute_value).

NormalizedCandidate.metadata contains additional OEM fields. Mercedes stores elective option codes there for denormalize(). Stellantis and Audi copy the columns selected by metadata_cols. A consumer that requires a metadata field must configure the loader and normalizer to supply it.

OEM behavior​

NormalizerCandidate identifierInventory identifierSource label
BMWcandidate_code_idorder_idBMW
Mercedescandidate_code_idorder_idNETSTAR for candidates; INVENTORY for inventory
Stellantisinventory_idinventory_idSTELLANTIS
Audiinventory_idinventory_idAUDI

BMW and Stellantis build candidate pools from DataFrame columns. Audi inherits the Stellantis implementation. Mercedes validates individual rows and builds a pool from the resulting objects.

multi_valued_attribute_types identifies types that may have several attribute codes. For other types, the columnar normalizers reject candidates with multiple attribute rows. When Stellantis, Mercedes and Audi read a nested vehicle_attributes column, they first remove duplicate rows with the same type and code. When they read a separate candidate_attributes table, they remove no rows. featurization/inventory has one row per (inventory_id, attribute_type, attribute_value), so a duplicate row there is an error. BMW preserves duplicate rows for this check.

Stellantis and Audi log malformed inventory attributes and skip the affected vehicle. Candidate validation failures raise exceptions.

Adding a normalizer​

  1. Implement the DataFrame methods in CandidateNormalizer. Implement individual-row methods when needed, or raise NotImplementedError.
  2. Declare a type: Literal["..."] field on the Pydantic class.
  3. For columnar inputs, convert source columns to the names expected by build_pool. For inputs that require individual-row validation, use CandidatePool.from_candidates().
  4. Add the class to AnyCandidateNormalizer and set candidates_normalizer.type in the OEM configuration.