Skip to main content

Featurization Checks

Two asset checks measure whether featurization gives the take-rate, demand and candidate tables the attributes they read.

Required attribute types​

The required attribute checks on featurization/inventory and featurization/synthetic measure, by weight, the vehicles that have no value of a type every vehicle must have, such as PAINT_COLOR. Such a vehicle has the value NO_<type> or no row of the type. Take rates and demand count NO_<type> as if a buyer chose it.

required_attribute_check of VehicleFeaturizationComponent names the types. Each must be a type that a multiclass scheme assigns. Every OEM with a VehicleFeaturizationComponent sets it.

Each vehicle counts with a weight, so that the checks judge the vehicles the take rates read:

  • a vehicle in stock (is_in_inventory) and a synthetic candidate weigh 1;
  • a vehicle sold in the window weighs its recency weight of half_life, measured to the partition date;
  • any other vehicle weighs 0 and is not counted.

An inventory vehicle without a row in consolidated/inventory fails the check run.

half_life is required: the half-life in days, 30, 90 or 180. It sets how fast old sales stop counting and need not match the half-life of any take-rate statistic. The checks read is_in_inventory, is_sold_in_window and a Date column inferred_sold_date from consolidated/inventory.

required_attribute_types_present, on each table, has WARN severity. It fails when the vehicles that lack a type hold more than max_missing_share of the table's weight, or when the table has no counted vehicle in the partition.

required_attribute_types_by_model, on featurization/inventory, has WARN severity. For each model and type, it divides the weight of the model's inventory vehicles that lack the type by the weight of all inventory vehicles. Vehicles with a null model_id count as one model. The check fails when this contribution exceeds max_model_contribution for any model and type. A model that holds 30% of the weight and lacks a type on 4% of it contributes 1.2%. A model that holds 0.1% of the weight and lacks the type on every vehicle contributes 0.1%. The metadata lists the models with the highest contributions.

A missing value has one cause:

CauseMeaningWhere to fix it
no_modelthe vehicle's model_id is null or not in consolidated/modelsthe consolidated inventory's model foreign key
catalog_has_typea featurized feature of the model has the type, but the vehicle's feature_ids lack itthe consolidated inventory's feature foreign keys
catalog_lacks_typeno featurized feature of the model has the typethe schemes

A vehicle is also counted as catalog_lacks_type when TypeSafe answered its model's feature with the scheme's outside option, a value the scheme does not list. The featurization tables do not record that answer. When a model's vehicles usually get a type from an inventory_attributes column, a vehicle missing that type is still counted as catalog_lacks_type, because the check reads only the featurized features.