Skip to main content

Branch Deployments

A pull request gets a Dagster+ branch deployment while it is open and has the branch-deploy label. The deployment reads and writes Iceberg tables on a Nessie branch named after the git branch, with each / replaced by -.

Deploying​

The deploy job of ci.yml deploys the pull request's affected code locations after CI passes. Adding the label to a pull request starts a CI run, which deploys the current commit.

Before it deploys, CI reads the pull request's current labels and state. CI skips the deployment when the pull request was closed or lost the label while CI ran.

The build_image job pushes to the ai-<oem>-branch ECR repositories, never to the prod repositories, and tags each image with the branch name. The deploy job points each code location at its -branch repository. Both jobs use the credentials of the aws-branch-deploy environment. See CI Credentials.

Code servers​

The Dagster agent stops a branch deployment's code server after the server serves no request for 2 hours. The next request starts the server again. BranchDeploymentServerTtlSeconds in deployments/aws/cloudformation/ecs-agent-vpc-private.yaml sets the interval.

Teardown​

branch-deployment-cleanup.yml deletes the branch deployment when a labeled pull request closes or loses the label. It also deletes the Nessie branch. When the pull request was merged, it first merges the Nessie branch into main.

GitHub starts no pull_request run for a pull request that has merge conflicts. The workflow therefore runs on pull_request_target. GitHub runs that workflow file from the base branch, so a change to the workflow takes effect only after the pull request that changes it is merged.

The workflow skips a pull request from a fork. The workflow holds production secrets, and a fork's branch name can match the name of a Nessie branch of this repository.

stale-branch-deployment-cleanup.yml runs every day. It deletes each branch deployment whose pull request is closed or does not have the label. It also expires the deployment of an open, labeled pull request when no code location of the deployment was updated for 14 days. Expiring a deployment does the following:

  1. Deletes the branch deployment.
  2. Deletes the Nessie branch, and with it the data that the branch deployment wrote.
  3. Removes the branch-deploy label.
  4. Comments on the pull request.

Add the label again to deploy the pull request to a new branch deployment, on a new Nessie branch from main.

The workflow runs only on its schedule. See CI Credentials. To list what the daily cleanup would delete, log in with dg plus login and run the script with --dry-run:

uv run scripts/delete_stale_branch_deployments.py --ttl-days 14 --dry-run

The script's docstring lists the environment variables it reads.