Featurization Checks
Two asset checks measure whether featurization gives the take-rate, demand and candidate tables the attributes they read.
Required attribute types
The required attribute checks on featurization/inventory and featurization/synthetic measure,
by weight, the vehicles that have no value of a type every vehicle must have, such as PAINT_COLOR. Such a vehicle has
the value NO_<type> or no row of the type. Take rates and demand count NO_<type> as if a buyer
chose it.
required_attribute_check of VehicleFeaturizationComponent names the types. Each must be a type
that a multiclass scheme assigns. Every OEM with a VehicleFeaturizationComponent sets it.
Each vehicle counts with a weight, so that the checks judge the vehicles the take rates read:
- a vehicle in stock (
is_in_inventory) and a synthetic candidate weigh 1; - a vehicle sold in the window weighs its recency weight of
half_life, measured to the partition date; - any other vehicle weighs 0 and is not counted.
An inventory vehicle without a row in consolidated/inventory fails the check run.
half_life is required: the half-life in days, 30, 90 or 180. It sets how fast old sales
stop counting and need not match the half-life of any take-rate statistic. The checks read
is_in_inventory, is_sold_in_window and a Date column inferred_sold_date from
consolidated/inventory.
required_attribute_types_present, on each table, has WARN severity. It fails when the
vehicles that lack a type hold more than max_missing_share of the table's weight, or when the
table has no counted vehicle in the partition.
required_attribute_types_by_model, on featurization/inventory, has WARN severity. For each model
and type, it divides the weight of the model's inventory vehicles that lack the type by the weight
of all inventory vehicles. Vehicles with a null model_id count as one model. The check fails when
this contribution exceeds max_model_contribution for any model and type. A model that holds 30% of
the weight and lacks a type on 4% of it contributes 1.2%. A model that holds 0.1% of the weight and
lacks the type on every vehicle contributes 0.1%. The metadata lists the models with the highest
contributions.
A missing value has one cause:
| Cause | Meaning | Where to fix it |
|---|---|---|
no_model | the vehicle's model_id is null or not in consolidated/models | the consolidated inventory's model foreign key |
catalog_has_type | a featurized feature of the model has the type, but the vehicle's feature_ids lack it | the consolidated inventory's feature foreign keys |
catalog_lacks_type | no featurized feature of the model has the type | the schemes |
A vehicle is also counted as catalog_lacks_type when TypeSafe answered its model's feature with
the scheme's outside option, a value the scheme does not list. The featurization tables do not
record that answer.
When a model's vehicles usually get a type from an inventory_attributes column, a vehicle missing
that type is still counted as catalog_lacks_type, because the check reads only the featurized
features.