Skip to main content

Simulated Runs

SimulatedRunReportComponent posts a daily Slack report of an OEM's simulated ordering runs. Each Dagster run of the TemporalWorkflowComponent's job starts one SimulatedOrderingRunWorkflow for one as_of_date, with every dealer it selects. The report covers the runs of one as_of_date.

Job​

<oem>_<market>_simulated_run_report_job posts the report. It has one partition per as_of_date and defines no asset.

In each Dagster run, the simulation_job op tries to start one workflow for the partition. The workflow's dealer_ids holds every dealer the op selects. The workflow id is the run_id that the ordering_runs document of each of those dealers records. The op writes three maps to its output metadata:

  • workflow_ids: the workflow id of each dealer whose workflow started.
  • launch_failures: the start error of each dealer whose workflow failed to start. When the workflow was already started, it also holds each selected dealer that the workflow was started without.
  • stale_dealers: the latest collection date of each dealer that the op started no workflow for, or null when the table has no collection date for the dealer.

When the op records a launch failure, it writes its output and then fails, so a failed simulation_job run still writes all three maps.

The report job reads the three maps from every successful or failed simulation_job run of the partition. A dealer whose workflow failed to start in one Dagster run and started in a later one counts as started. When several Dagster runs of the partition started a workflow for one dealer, as a rerun does, the job reports the workflow of the latest of them. The job reads each dealer's document of each started run from the ordering_runs collection. From volume_optimizations, it reads the volume optimization whose volumes set the run's targets. It reads each dealer's name and consolidated dealer_id from consolidated/dealers at the same partition, through the Iceberg IO manager. The dealer_id selects the dealer's row of run_metrics. A run_metrics row with no dealer_id belongs to a run with one dealer, as every run written before that column had.

A run writes its ordering_runs document only after it builds its targets, or when it stops early. A run that never writes one is still counted, and its reply says "No run document". A dealer whose workflow failed to start is counted as not started, and its reply shows the error. The job fails for a partition whose simulation_job runs recorded no workflow_ids.

The job posts one message to slack_channel with:

  • a link to <report_url>/dashboard. The link sets the query parameter oem to the job's OEM. It sets market to ordering_market, or to the market of the defs directory when ordering_market is unset. It sets startDate and endDate to the as_of_date, and runMode to simulated;
  • the number of runs for each status, of runs with no run document and of runs not started. The skipped count includes the dealers in stale_dealers;
  • the recommended volume of the completed runs;
  • the mean weighted_misalignment_reduction_rate of the completed runs that have a rate;
  • the number of completed runs with no targets.

The job then posts one reply in the message's thread for each run, except skipped runs and completed runs with no targets. A reply shows the run's dealer, report link and status. The status of a run that is neither completed nor still running includes its exit reason. When the run has targets, the reply also lists:

  • the recommended volume and the number of models it covers;
  • the recommendation count, with the failed and unreached targets;
  • weighted_misalignment_reduction_rate and overpruned_rate, from the run's partition of <oem>/<market>/reporting/run_metrics (see Run Metrics);
  • run_duration_seconds and the phase in phase_durations with the most seconds, from the same partition. The reply omits this item when run_duration_seconds is null.

After those replies, the job posts one reply for each group of runs that has members, listing the group's dealers:

  • completed runs with no targets;
  • skipped runs, one group for each exit reason. The group for no recent inventory collection also lists the dealers in stale_dealers, with the date of each dealer's latest collection;
  • runs with no run document;
  • runs not started, with the error of each.

The report link opens <report_url>/reports/<run_id>?dealer=<dealer_code> on the reporting dashboard, which signs the dealer's report_uri on each request. A run document with no report_uri shows "No report". A run document with no exit_reason shows "unknown reason". A run with no materialized run_metrics partition shows "Metrics: not computed".

If the job fails after it posts the message, delete the incomplete thread before you re-execute the Dagster run. A re-execution posts the message and every reply again. A manual run of the job posts for its partition, whether or not the partition was posted before.

Sensor​

The sensor reads the successful and failed simulation_job runs created in the last lookback_days days. It checks each of their partitions on or after start_date that has no successful run of the report job and has a simulation_job run with workflow_ids. It requests a run of the report job for each such partition, with the partition as its run_key, when:

  • settle_minutes have passed since the latest simulation_job run of the partition with workflow_ids ended;
  • every recorded run has an ordering_runs document with a status other than RUNNING, and settle_minutes have passed since the last of them was updated;
  • every run with an ordering_runs document has a materialized run_metrics partition, or metrics_wait_minutes have passed since the last run was updated.

A completed run sets its report_uri after it writes its final status. The settle_minutes wait after the last update gives it time to do so.

When cutoff_hours have passed since that simulation_job run ended, the sensor requests a run of the report job without checking the simulated runs. On a branch deployment the sensor is stopped by default.

Configuration​

type: ai_dagster.components.SimulatedRunReportComponent
template_vars_module: ai_stellantis.defs.stellantis.us.template_vars
attributes:
start_date: "2026-10-23"
simulation_job: stellantis_us_ordering_job
label: Stellantis US
slack_channel: "{{ slack_channel }}"
report_url: https://ppm-reporting.app.autointel.ai

settle_minutes defaults to 15, metrics_wait_minutes to 60, cutoff_hours to 8 and lookback_days to 3.

ordering_market is the market of the simulated runs. The dashboard filters runs by this market. Set ordering_market when the market of the runs differs from the market of the defs directory. The Mercedes defs directory is mercedes/mbusa, and its simulated runs have the market us, so it sets ordering_market: us.