Cost Attribution

Every AWS resource this repo creates carries autointel:* tags, so spend can be queried per project and per part of the platform instead of read off a console. org.vendors.aws.weekly_platform_cost_breakdown and monthly_platform_cost_breakdown are written those amounts in Iceberg.

The tables

Every query behind them is bounded to autointel:platform, so a period sums to what this repo costs rather than to the account bill. One row per (oem, market, group), plus one row with all three null holding platform spend that carries no OEM tag — Nessie, NAT, the agent.

Column	Meaning
oem / market / group	The tag values; null on the unallocated row
prod_unblended_usd / prod_amortized_usd	Spend from tasks launched by the production deployment
branch_unblended_usd / branch_amortized_usd	Spend from tasks launched by a branch deployment
untagged_unblended_usd / untagged_amortized_usd	Spend no deployment can claim
_partition_date	Start of the week or month

The untagged_ columns are not a gap to be closed. Only tasks our own code launches can read the deployment at launch time, so code servers, run coordinators, the per-project buckets and registries, and all shared infrastructure are recorded there permanently. For any row, prod + branch + untagged equals its unfiltered platform total.

Dagster on its own — everything except the ordering service — is platform = ppm-data-platform AND NOT group = ordering, since ordering is the only non-Dagster group.

The platform_share_of_account check on each asset reports what fraction of the account bill the platform filter accounts for. A fall in that share means a resource went untagged, not that spend disappeared.

The breakdown stops at (period, oem, market, group). Assets within one group carry identical tags, so individual assets are indistinguishable — see per-asset cost below.

The monthly table is a separate read rather than a rollup of the weekly one: weeks straddle month boundaries, so summed weeks cannot produce a calendar month. Cost Explorer also reconciles a month to the invoice in a way summed sub-periods do not.

How the three scopes are read

Cost Explorer caps GroupBy at two entries and there are four tag dimensions to cover, so OEM and deployment are handled by filtering instead. The OEM values present in the period are enumerated first, then each is read three times grouped by [market, group]:

platform AND oem=<x> AND deployment=prod      → prod
platform AND oem=<x> AND deployment ABSENT    → untagged
platform AND oem=<x>                          → total

branch is the remainder — total less the other two — rather than its own query, because deployment values are unbounded (one per open pull request) and enumerating them would cost a request each. That also makes the three columns reconcile to the row's platform total by construction.

Restatements

AWS amounts move as a period settles:

• estimated while it is open
• base usage finalized when the invoice issues on days 1-5 of the following month
• support fees and final tiers at true close on days 6-7

So every partition is read twice:

Asset	First read	Restated
weekly	Sunday 03:00 UTC, week just closed	the following Sunday's tick
monthly	the 1st, 04:00 UTC	the 8th, first day past true close

Anything older than that window stays frozen. AWS can restate for 12-36 months via backfills and tag corrections. Catching those needs a manual backfill.

Getting per-asset cost

The tags cannot answer it, but one step task is one asset materialization, and each asset's ecs/cpu and ecs/memory op tags fix its task size. So per-run compute cost is Fargate rates times duration, which Dagster+ reports:

vCPU × hours × $0.04048  +  GB × hours × $0.004445     (us-east-1; confirm current rates)

That covers compute only — an asset's share of S3, NAT and Nessie is not attributable to it.

Tag keys

Key	Values	Meaning
autointel:platform	ppm-data-platform	Bounds a query to this repo's spend. Carried by every resource, in code and in Terraform.
autointel:oem	project slug — bmw, mercedes, org, public, …	The project the resource belongs to. Matches the code location name without the ai_ prefix, and local.oems in Terraform.
autointel:market	us, mbusa, mena, …	Market, on step tasks that process one.
autointel:group	raw, transformed, consolidated, days_to_lot, warehouse, registry, nessie, network, agent, code_server, run, …	Which part of the platform. DAG layer for step tasks, infrastructure role for everything else.
autointel:deployment	prod, or a git branch name	Which deployment launched the task. Only tasks our own code launches can carry it.

Where each tag is set

Surface	Mechanism	Dimensions
Step tasks (bulk of compute)	Executor config set in build_core_defs() (ai_dagster/core_defs.py), refined per asset by build_ecs_task_tags() (ai_dagster/components/common.py)	platform, oem, deployment, plus market and group where the component knows them
Code servers, run coordinators	projects/*/container_context.yaml → ecs.server_ecs_tags / ecs.run_ecs_tags	platform, oem, group
Nessie, ALB, ECR, S3, DynamoDB, VPC endpoints, Lambda, log groups	provider.default_tags plus per-resource tags in deployments/aws/terraform/solutions/dagster-agent/	platform, group, and oem on the per-project buckets and registries
ai_ordering service	deployments/aws/terraform/solutions/ai-ordering/provider.tf	platform, group
VPC, NAT gateways, agent cluster and service, Cloud Map	tags on aws_cloudformation_stack.dagster_agent, propagated by CloudFormation to the resources it creates	platform, group

Two details decide whether a tag reaches the resource that is actually billed:

• Fargate charges accrue against the task, not the service. An ECS service must set propagate_tags = "SERVICE" (PropagateTags: SERVICE in CloudFormation) or its tags never reach the task. Propagation applies to tasks launched after the change, so an already-running task stays untagged until it is replaced.
• A step task's own tags replace the executor's rather than merging with them. The ecs_executor shallow-merges ecs/run_task_kwargs from op tags over its own config, so a callsite that sets tags without oem would drop attribution that the executor would otherwise have supplied. build_ecs_task_tags() emits the executor-level dimensions alongside market and group for exactly this reason. A component with no OEM to name should set no ECS task tags at all and inherit the executor's.

Activation

A tag key is not a Cost Explorer dimension until it is activated. Setting the tag on resources is not enough.

var.cost_allocation_tag_keys in the dagster-agent solution is the source of truth: keys in that list are held Active, and removing one deactivates it. Adding a key takes two changes: one that starts emitting it, then one that activates it. AWS rejects activation of a key it has not yet observed, and takes up to 24 hours to report a new one. Check what it can see with aws ce list-cost-allocation-tags --status Inactive.

Activation is not retroactive: only usage recorded after it can be attributed to the key.

Coverage

Reachable by tags:

• Fargate compute (step tasks, run coordinators, code servers, Nessie, ordering)
• S3 storage and requests
• ECR storage
• DynamoDB
• NAT gateway hours
• VPC endpoint hours
• ALB hours and LCUs
• CloudWatch Logs
• Lambda
• Cloud Map

Not reachable, by AWS design:

• tax
• support fees
• some Marketplace charges
• the portion of data transfer that AWS books against no taggable resource

Dagster+ is a SaaS subscription rather than an AWS charge, so it never appears here at all.

What the deployment dimension can and cannot split

Only tasks launched by our own code read the deployment from the environment as they launch. The Dagster+ agent's dagster.yaml has no per-deployment scoping, and container_context.yaml is shared by production and branch deployments, so:

Charge	Splits prod from branch
Step tasks	Yes
Code servers	No — a branch deployment runs its own set for the life of the pull request
Run coordinators	No
S3 warehouse	No — Nessie branches write to the same per-project bucket
Nessie, NAT, ALB, VPC endpoints, DynamoDB	No — one shared instance serves every deployment

Splitting the remainder means resource-level Cost and Usage Report data, where task ARNs and service names encode the deployment.

Verifying tags were applied

# A step task, while it runs or shortly after
aws ecs list-tags-for-resource --resource-arn <task-arn>

# A per-project warehouse bucket
aws s3api get-bucket-tagging --bucket ai-app-bmw-iceberg-prod

# Values AWS has observed for a key, once the task role has ce:GetTags
aws ce get-tags --tag-key autointel:platform \
  --time-period Start=2026-08-01,End=2026-08-08

aws ce get-tags returning an empty list for a key that is set on live resources means the key is not activated yet.

Cost of querying

Cost Explorer bills per request. A partition costs:

• three requests per OEM — one per directly queryable scope, plus the unbounded read branch is derived from
• four requests for the platform totals and the coverage check
• one GetTags call to enumerate the OEM values

At weekly and monthly schedule with restatement reads that comes to a few dollars a month. Finer partitions multiply it directly: a daily asset re-reading a trailing window costs about seven times as much as a weekly one.

What the assets need to run

• Every autointel:* key in var.cost_allocation_tag_keys Active in AWS Billing. An inactive key is not a GroupBy or Filter dimension, so filtering on it matches nothing and the table fills with zero-amount rows rather than failing.
• ce:GetCostAndUsage and ce:GetTags on the user-code task role, granted by the ecs-task-tag-read policy in deployments/aws/cloudformation/ecs-agent-vpc-private.yaml.