Skip to main content

BMWUSA Build Your Own (BYO) Source

Overview​

Public, unauthenticated API serving model catalog and feature/option data for BMW USA. Raw + transformed tiers only.

API​

Base URL: http://qa.configure.bmwusa.com/UBYOConfigurator Auth: None (public) Schedule: Daily at 08:00 UTC / 04:00 EDT (0 8 * * *). BYO catalog refreshes 02:00–03:00 EDT (06:00–07:00 UTC); the schedule runs >=1h after refresh completes.

EndpointPurpose
GET /v2/static/typelist?full=trueAll product codes with series/bodystyle metadata
GET /v4/BM/model/{productCode}Model detail (specs, pricing, flags)
GET /v4/BM/techspecs/{productCode}Nested techspecs (engine, transmission, performance)
GET /v4/BM/options/{productCode}Features/options array per model
GET /v4/configuration/start/{productCode}?trustedSource=trueConfigurator session — which features are configurable

Assets​

One raw asset per API endpoint, each with a corresponding transformed asset.

Dependency chain (raw):

  • Typelist: no upstream (fetches the full product array as a single raw row)
  • Models, Techspecs: upstream = typelist (reads product codes from the typelist body array)
  • Configuration, Options: upstream = models (reads product codes from _request_params)
Raw assetRaw source (ai_public_api.bmw.bmwusa_byo)Transformed entity
bmw/us/raw/bmwusa_byo_typelistTYPELISTBmwUsaTypelistEntity via extract_typelist
bmw/us/raw/bmwusa_byo_modelsMODELSBmwUsaModelEntity via extract_models
bmw/us/raw/bmwusa_byo_techspecsTECHSPECS(raw only — no transformed asset; consumed at consolidated)
bmw/us/raw/bmwusa_byo_configurationCONFIGURATIONBmwUsaFeatureEntity via extract_configuration (is_configurable=True)
bmw/us/raw/bmwusa_byo_optionsOPTIONSBmwUsaFeatureEntity via extract_options (is_configurable=None)

Transformed assets:

Transformed assetEntitySource key
bmw/us/transformed/bmwusa_byo_typelistBmwUsaTypelistEntity[bmwusa_byo, typelist]
bmw/us/transformed/bmwusa_byo_modelsBmwUsaModelEntity[bmwusa_byo, models]
bmw/us/transformed/bmwusa_byo_configurable_featuresBmwUsaFeatureEntity[bmwusa_byo, configuration]
bmw/us/transformed/bmwusa_byo_optionsBmwUsaFeatureEntity[bmwusa_byo, options]

Start dates​

Typelist, models, and options share byo_start_date (matches the 1-year inventory backfill horizon). Configuration uses byo_configuration_start_date — no legacy BQ data exists for this endpoint, so no backfill is possible.

Entity mapping​

Typelist → BmwUsaTypelistEntity​

Standalone entity in entities.py with validation_alias mappings. Extractor uses model_validate.

BYO fieldEntity fieldNotes
codemodel_codeProduct code (e.g. 267S)
agCodeag_code
namemodel_name
series.codeseries_codeNested object
series.nameseries_nameNested object
modelRangemodel_range_code
modelYearmodel_year
bodyStyle.codebody_type_codeNested object
bodyStyle.namebody_type_nameNested object
engineDriveTypedrive_type
sportsFlagsports_flag
fuelTypefuel_type
mPerformanceFlagm_performance_flag

Models → BmwUsaModelEntity​

Standalone entity in entities.py with validation_alias mappings. Extractor uses model_validate_json.

Series/bodyStyle fields are NOT in the model detail response — they come from the typelist and are joined at the consolidated tier.

BYO fieldEntity fieldNotes
codemodel_codeProduct code
modelRangemodel_range_code
namemodel_name
yearmodel_year
bodyStylebody_type_nameString (not nested object)
fuelTypefuel_typeSingle letter (E, D, G)
driveTraindrive_train
driveTypedrive_type
pricebase_price
horsepowerhorsepower
accelerationacceleration
sportsFlagsports_flag
mPerformanceFlagm_performance_flag
isElectrifiedis_electrified
isSdriveis_sdrive
isXdriveis_xdrive
destinationAndHandlingdestination_and_handling
modelRangeNamemodel_range

Features (options + configuration) → BmwUsaFeatureEntity​

Standalone entity in entities.py with validation_alias mappings. Extractor uses model_validate on the option dict; context-driven fields (model_code, brand, currency, is_configurable) are set post-validation.

BYO fieldEntity fieldNotes
codefeature_code
(from params)model_codeInjected from product code
namename
longDescriptiondescription
typefeature_type
familyCodefamily_code
isStandardis_standard
listPriceprice_gross
isPackageis_package
isAccessoryis_accessory
(derived)is_configurableTrue from configuration; None from options
combinationPricescombination_prices[{combFeatCode, combPrice}] → [CombinationPrice]
(flattened)parent_package_codeSet when feature is a package alternative
(flattened)slot_defaultCode of the default alternative in the slot

Key fields: feature_code, model_code, parent_package_code.

Package flattening​

Packages with packageContent.packageSlots are flattened to one row per alternative:

  • Package row: feature_code="S0ZPK", parent_package_code=None, is_package=True
  • Alternative rows: feature_code="S0776", parent_package_code="S0ZPK", slot_default="S0776" (first alternative = default)

Standalone features that also appear inside packages get both a standalone row and a row per package appearance.

Proxy-code slot re-listing​

Some option codes are not a wheel themselves but always add a corresponding wheel — selecting S0ZWH adds wheel S01U0 on the affected models. A package slot built around the proxy therefore offers the same choice as the slot built around the wheel it adds. Both the options endpoint (loadtype=full) and the configuration endpoint (trustedSource=true) return hidden package content, so both slots come back. Without handling, one feature appears in two slots of the same package with conflicting slot_default, which appears as a discrepancies check failure on bmwusa_byo_features and bmwusa_byo_configurable_features.

_drop_relisted_slots resolves this at extract time: two slots whose alternative code sets differ by exactly one code on each side offer the same choice when one of those codes is a proxy for the other, so the proxy copy is dropped. Observed structure for model 274M, package S0ZJJ:

copyalternativesslot_default
proxy (dropped)S01T8, S01T9, S01U1, S0ZWHS01T8
wheel (kept)S01U0, S01T8, S01T9, S01U1S01U0

_PROXY_OPTION_CODES holds the known proxy codes. It is a single entry today (S0ZWH); add to it if another proxy code turns up, rather than broadening the matching rule.

Two properties worth preserving if this is ever revisited:

  • Set comparison, not position. BMW's packageSlots order is not stable between fetches, and the proxy copy is not reliably last — in 274M it is at index 0. Any positional rule ("take the first slot") picks a different winner on different days.
  • Pair-scoped. A slot is only dropped once its wheel twin is found in the same package, so a package exposing only the proxy copy keeps it. The filter never causes a net loss of slots.

Identical slot re-listing​

A package can also list the same slot twice, with the alternative codes in a different order. The first alternative sets slot_default, so each copy gives the codes a different slot_default. Observed for model 27XG, package S0ZMP:

copyalternativesslot_default
droppedS03QQ ($800), S03QY, S03QN, S03QT, S03QU, S03QE, S03QGS03QQ
keptS03QY ($0), S03QQ, S03QN, S03QT, S03QU, S03QE, S03QGS03QY

_drop_relisted_slots keeps the copy whose first alternative has the lowest price. On a price tie, it keeps the copy whose first alternative has the lowest code. In S0ZMP the kept default is the $0 wheel S03QY. The package also lists S0PKG31 (21" M Polygon Wheels), whose description matches S03QY.

Configurable derivation​

Options and configuration produce separate assets at the transformed tier. At the consolidated tier, the two are unioned on (feature_code, model_code, parent_package_code). The options asset has is_configurable=None; the configuration asset has is_configurable=True. The tiebreak picks non-null, so any feature present in both gets is_configurable=True in the consolidated output.

The configuration extractor collects all feature codes from the configuration/start response blocks (designs, tops, colors, wheels, upholstery, trims, packages, featuredPackages, vehiclePrograms as lists; options, accessories as dicts of lists), then looks up each code in optionDetails to build full feature records.

Feature type derivation​

The consolidated tier derives feature_type, a FeatureType value, for each feature. BYO's type field holds a one-letter code. The transformed tier stores it as feature_type, and the consolidated adapter renames it to category:

typeFeatures
Soptions, packages and wheels
Ppaints
IBMW Individual paints
Fupholstery
nullS0PKG… bundles and ACC_… service plans

S covers most of the catalog, so the derivation also reads family_code, which holds BYO's familyCode:

family_codefeature_type
FELGEWHEEL
LEIST, ILEISINTERIOR_TRIM
IHIMMHEADLINER
MET, UNI, ILACKPAINT

The derivation checks these rules in order, and the first that holds sets the type:

  1. is_package gives PACKAGE.
  2. Any of these gives ACCESSORY:
    • is_accessory is true.
    • The code starts with ACC_ or US_. US_ codes are dealer-installed accessories, such as M Performance Parts, floor liners and complete wheel sets.
    • The family is ACC_AFT.
  3. Family FELGE gives WHEEL. A name that contains two consecutive digits, such as a wheel size, and contains "wheel" but not "steering" also gives WHEEL, whatever the family.
  4. Any other family in the table above gives that family's type.
  5. A P or I type gives PAINT, and an F type gives UPHOLSTERY.
  6. Every other feature is an OPTION.

BQ backfill​

Legacy tables in app-template-382723 — the project gcp-dataform-inventory declares as the source of record. A same-named pair exists in ai-app-bmw; do not use it. It is missing model codes (156 vs 202 distinct code for 2026-06-25) and truncates package_content.packageSlots[].alternatives to one entry per slot, so it looks plausible while silently under-reporting.

TableBackfill targetReshape
product.raw_model_metadatabmwusa_byo_typelist_reshape_byo_typelist
product.raw_model_metadatabmwusa_byo_models_reshape_byo_model
product.raw_model_metadatabmwusa_byo_techspecs_reshape_byo_techspecs
product.raw_feature_metadatabmwusa_byo_options_reshape_byo_options

No backfill for configuration — the legacy scraper did not call the configuration/start endpoint.

The model metadata table (~272k rows) serves three backfill targets. Each reshape reverses the legacy scraper's flat snake_case storage back to camelCase BYO response format. The techspecs reshape unflattens __-separated columns (e.g. engine__combined_horse_power) into the nested dict the API originally returned.

The feature metadata table (~42M rows) uses ByoFeatureBackfillReader, which groups rows by product_code with ARRAY_AGG before reshaping.

Duplicate source rows​

The legacy collector runs twice daily, and both tables are append-per-fetch, so every entity has exactly two rows per date (rows / distinct code = 2.0 on every partition checked). bq_dedup_on keeps the newest row per entity — [code] for model metadata, [product_code, code] for feature metadata. For the feature reader the dedup sits inside the ARRAY_AGG, since GROUP BY product_code would otherwise collapse the duplicates into one row with every feature listed twice.

Structure parity with live​

typelist is a single /v2/static/typelist call stored as one row holding the whole array, so its backfill uses ByoTypelistBackfillReader to aggregate the partition into one row. Emitting one row per model gives backfilled partitions a structure no live partition has, breaking raw row counts and the row-count-deviation check. models and techspecs are genuinely per-code (/v4/BM/model/{code}, /v4/BM/techspecs/{code}), so one row per model is correct for those.

Fidelity limit: hidden package content is not recoverable​

The legacy collector called /v4/BM/options/{code} without loadtype=full (byo.ts getFeatures), so hidden package content was never captured. Measured against the live endpoint for 262V:

slotsalternatives
default (what the backfill has)5483
loadtype=full (what the live asset fetches)88182

Backfilled bmwusa_byo_features therefore runs roughly 73% of live on package children (mean 250 vs 342 features per model across 176 shared model codes; 172 of 176 lower, none higher). That is a property of the source, not a bug — do not compare feature counts across the backfill boundary. It also means _drop_relisted_slots is inert on backfilled partitions: the proxy/wheel duplicate slot pairs exist only in hidden content.

One further gap, unrelated to loadtype: raw_model_metadata holds 181 codes where typelist lists 182, because getProduct returns null when /v4/BM/model/{code} fails and drops the product. Probing all 182 codes found exactly one 404 (262X).

VDML vs BYO decision​

The legacy Dataform pipeline unions VDML and BYO data with VDML winning per model_code. Analysis shows VDML adds no unique value for active inventory:

  • Models: 302 VDML-only model codes are all retired years (2018–2023). BYO covers all 486 active models.
  • Features: 770 VDML-only feature codes are from those same retired years. BYO covers all active features.
  • Fields: VDML's unique fields (productClass, salesGroup) are not consumed in the pipeline. productClass ≡ BYO's familyCode (which is used). salesGroup exists in BYO's raw_feature_metadata but is unused.

Decision: Ingest BYO only. Feature-type classification that VDML provides via pre-typed arrays is handled at the transformed tier using BYO's type field.