Candidate pool
A scope's candidates are held as columns (CandidatePool), not as
list[NormalizedCandidate]. The Stellantis and BMW normalizers emit those
columns directly, so no object exists between the source frame and the pool.
Mercedes validates per row and assembles the pool from the objects — see
Construction.
NormalizedCandidate remains the boundary type, and the boundary is the
recommendation: strategies.py calls materialize for the handful of
candidates a round selects, and that is what services/ai_ordering receives.
Why
At national scope the candidates are the largest thing an ordering run holds. One
Pydantic model per candidate plus one per attribute, for 50,000 candidates of 18
attributes, measures 656 MB resident. A Lambda invocation was killed at
Max Memory Used: 8192 MB of 8192.
What it costs
The same candidates both ways — ids, attributes, option codes, feature ids and metadata included:
| candidates | objects (getsizeof) | objects (peak RSS) | pool | reduction |
|---|---|---|---|---|
| 10,000 | 72.9 MB | 130.7 MB | 4.3 MB | 17.0x / 30.5x |
| 50,000 | 364.6 MB | 655.8 MB | 21.4 MB | 17.0x / 30.6x |
Two instruments, because they answer different questions. A recursive
sys.getsizeof is what a test can assert. Peak RSS is what the Lambda runs out
of, and it is higher because getsizeof sees neither allocator overhead nor
Pydantic's internals. Plan against that column. Both scale linearly.
estimated_size() reports frame buffers. It excludes the category map the
encoded columns index into, which polars shares across columns and which is
~1.6 MB for a 50,000-candidate pool. Sizing that map per column overstates it
several times over: Series.dtype.categories.to_series() returns the shared map, so it
reports every category in the pool for a column holding one distinct value.
Structure
One long frame per thing a candidate has many of — attributes, options,
features — against scalars, which carries one row per candidate and its
metadata. Long form is what makes the reduction: the repeated ids and codes
dictionary-encode.
Encoding is applied where values repeat and skipped where they do not. A column
with one row per candidate is all-distinct, so encoding it costs indices on top
of a map holding every value. id on scalars is therefore unencoded, and
metadata shares that frame rather than carrying a second copy of the ids —
holding the id map once instead of per frame is worth about 2x on its own.
Option codes have their own frame rather than joining the attributes under
OPTION_CODE. An OPTION_CODE restriction matches a candidate's option codes,
while a candidate may separately carry an attribute of that type which it must
not match. Folding them together conflates two different questions.
A candidate's feature_ids hold every feature the vehicle has, as candidates_output
writes them, including features that come with an option, such as the black roof of a
two-tone paint package. Its options are only the features of
<oem>/<market>/output/order_form_features, the features that go on their model's order
form, so a candidate's option_ids hold the codes the ordering service submits. The
recommendation event therefore holds the features that featurization reads, and an
evaluation that featurizes a recommended candidate gives it the attributes its take rates
and references know it by.
attribute_pairs() returns text and encoded_attribute_pairs() returns the
encoded frame. Only consumers that join against frames loaded from the warehouse
need the text form — a Categorical-to-Utf8 join raises rather than coercing —
so pool-internal work is not decoded to be iterated.
Construction
An OEM normalizer renames its source frame to the canonical column set — id,
attributes, feature_ids, model_year, plus one column per metadata key — and
calls build_pool, which explodes the nested attribute column into the long
frames. normalizers/columnar.py takes no column-name parameters: reconciling
each source's column names is the normalizer's job, and by the time build_pool sees a frame
the names are ours.
build_pool removes repeated features within each candidate's feature_ids list before
it explodes the lists into the features frame. Each candidate id appears in one row of
the frame that build_pool receives, so this gives the same rows as removing repeated
(id, feature_id) pairs from the exploded frame. Removing them from the exploded frame
hashes every pair. feature_ids holds every feature of a vehicle, so the memory of that
hash grows with the number of features of every candidate in the scope.
When the table map names a candidate_attributes table, the attributes are never
nested. The assembler reads the table's flat (inventory_id, attribute_type, attribute_value) rows. Inside the scan, it keeps the rows of the scope's candidates
whose attribute_type is not null, and casts them to Categorical. It then passes the
frame to normalize_candidate_attributes.
When candidate_attributes and inventory_candidates name the same table, the
assembler scans the table once. Every OEM's table map names featurization/inventory
for both. The assembler reads the rows either frame keeps and splits each streamed
batch into the candidate attribute rows and the dealer's inventory rows. A table in
only one entry's extra_namespaces, such as featurization/synthetic for
candidate_attributes, is scanned once for that frame alone. A single
collect_all over two queries does not share the scan. Their filters differ, so
Polars plans two scans. The table is not sorted by model_id, so one scope's rows
are spread over a large share of the table's row groups. Each scan downloads all of
those row groups.
The scan reads the columns that either frame keeps or filters on, or every column
when one of the two table-map entries has no select_columns. A batch holds those
columns for every candidate attribute row, so the scan casts its string columns to
Categorical before the batch is split. The assembler casts the dealer's inventory
rows back to the table's types.
A large scope has tens of millions of these rows. Each of these steps would hold another copy of them:
- nesting the rows and exploding them again;
- removing duplicate rows as strings;
- checking exclusivity with an in-memory group-by.
Together, those copies put one scope's build above the ordering worker's memory
limit. This path therefore neither nests nor removes duplicate rows. Instead,
normalize_candidate_attributes raises ValueError when a row appears more than
once:
- for an exclusive type, the exclusivity check finds the repeated row;
- for a multi-valued type, a second check counts the rows of each candidate, type and value.
Both checks run on the streaming engine.
normalize_candidate_attributes sorts the rows by candidate and attribute type.
In the table's file order, one candidate's rows are spread across the table. The
pickled pool that an ordering activity returns then compresses several times worse.
For 50,000 candidates of 18 attributes, building the same pool both ways:
| Python allocated (peak) | frames held | |
|---|---|---|
| from objects | 747 MB | 19.8 MB |
| from frames | ~0 | 19.8 MB |
tracemalloc measures Python allocations, which is exactly what the object graph
is. The frames themselves are Rust-side and it does not see them, so read the
first column as "the object graph" and the second from estimated_size.
from_candidates stays, for the sources that validate per row: Mercedes reads
model_id off each row, parses six JSON-encoded fields, and sums attribute
prices into total_price.
Sharing one pool across scopes
The take-rate bulk path normalizes once per model_id and hands every dealer
scope of that model the same pool, paired with the ids that scope draws from it
(SharedCandidates). subset runs on first access to candidate_pool, so a
scope that is built but never scored costs its id list and nothing more.
A scored scope of 5,000 candidates holds a 1.98 MB subset pool, against one
per-model object graph removed. services/ai_ordering builds one scope per
invocation, so only a multi-scope caller sees the per-scope figure at all.
Scoring on the encoded attribute rows
AttributeTurnScorer and DemandSupplyGapScorer join encoded_attribute_pairs()
to their turn, demand, supply and weight tables. Before each join,
dictionary_encoded casts the string columns of the table to Categorical. The
scorers cast id to Utf8 only after grouping, when the frame has one row per
candidate.
A large scope has tens of millions of attribute rows. Cast to Utf8, those rows
take several times the memory of the encoded rows. Each join and group-by over the
Utf8 rows allocates another frame of that size. The resulting peak memory exceeds
the memory of a serverless ordering worker. A
scorer that needs attribute strings casts them to Utf8 only after it has reduced
the rows to the values it returns.