Branch Deployments
A pull request gets a Dagster+ branch deployment while it is open and has the
branch-deploy label. The deployment reads and writes Iceberg tables on a Nessie branch
named after the git branch, with each / replaced by -.
Deploying
The deploy job of ci.yml deploys the pull request's affected code locations after CI
passes. Adding the label to a pull request starts a CI run, which deploys the current commit.
Before it deploys, CI reads the pull request's current labels and state. CI skips the deployment when the pull request was closed or lost the label while CI ran.
The build_image job pushes to the ai-<oem>-branch ECR repositories, never to the prod
repositories, and tags each image with the branch name. The deploy job points each code
location at its -branch repository. Both jobs use the credentials of the
aws-branch-deploy environment. See CI Credentials.
Code servers
The Dagster agent stops a branch deployment's code server after the server serves no
request for 2 hours. The next request starts the server again.
BranchDeploymentServerTtlSeconds in deployments/aws/cloudformation/ecs-agent-vpc-private.yaml
sets the interval.
Teardown
branch-deployment-cleanup.yml deletes the branch deployment when a labeled pull request
closes or loses the label. It also deletes the Nessie branch. When the pull request was
merged, it first merges the Nessie branch into main.
GitHub starts no pull_request run for a pull request that has merge conflicts. The
workflow therefore runs on pull_request_target. GitHub runs that workflow file from the
base branch, so a change to the workflow takes effect only after the pull request that
changes it is merged.
The workflow skips a pull request from a fork. The workflow holds production secrets, and a fork's branch name can match the name of a Nessie branch of this repository.
stale-branch-deployment-cleanup.yml runs every day. It deletes each branch deployment
whose pull request is closed or does not have the label. It also expires the deployment
of an open, labeled pull request when no code location of the deployment was updated for
14 days.
Expiring a deployment does the following:
- Deletes the branch deployment.
- Deletes the Nessie branch, and with it the data that the branch deployment wrote.
- Removes the
branch-deploylabel. - Comments on the pull request.
Add the label again to deploy the pull request to a new branch deployment, on a new
Nessie branch from main.
The workflow runs only on its schedule. See CI Credentials. To
list what the daily cleanup would delete, log in with dg plus login and run the script
with --dry-run:
uv run scripts/delete_stale_branch_deployments.py --ttl-days 14 --dry-run
The script's docstring lists the environment variables it reads.