Skip to main content

Valhalla routing tiles

osm/us/valhalla/tiles is a Valhalla routing graph of the US for pyvalhalla. It is built from Geofabrik's OpenStreetMap extract of the US and stored as one tile archive. The archive is the indexed .tar file that valhalla.Actor reads from the path in its mjolnir.tile_extract config setting.

Asset​

The asset is defined in defs/osm/us/valhalla_tiles/defs.yaml of ai_public. The component's country sets the key osm/<country>/valhalla/tiles, and its region names the country's Geofabrik extract, such as north-america/us.

It is not an Iceberg table. Each materialization writes one object to the ai-app-public-artifacts-prod bucket, at a path that includes the asset key:

s3://ai-app-public-artifacts-prod/osm/us/valhalla/tiles/<tiles_version>/valhalla_tiles.tar

On a branch deployment the path starts with branches/<git branch>/, so a materialization on a branch deployment never writes a version that prod reads.

A published tiles_version never changes. The upload uses exclusive creation (mode="create"), which fails when an object already exists at the path.

When an object already exists at the path, the asset records a materialization of that object without downloading or building. It sets archive_reused to true in the materialization's metadata.

The existing object is a complete archive that passed the isochrone check, step 4 under Build. S3 makes an object visible only after its upload completes, and the asset uploads the archive only after the check.

A run can end after the upload but before Dagster records the materialization, for example when the run reaches its dagster/max_runtime limit. Launching the partition again records a materialization of the uploaded archive.

The bucket holds files that ai_public builds and that are not Iceberg tables. Terraform defines it in deployments/aws/terraform/solutions/dagster-agent/artifacts_s3.tf, with these lifecycle rules:

  • a multipart upload still incomplete after 7 days is aborted;
  • an object under branches/ expires after 30 days.

The user-code task role's public-artifacts-s3-access policy, in deployments/aws/cloudformation/ecs-agent-vpc-private.yaml, allows:

  • s3:GetObject, s3:PutObject and s3:AbortMultipartUpload on the bucket's objects;
  • s3:ListBucket and s3:GetBucketLocation on the bucket.

s3fs uploads the archive in parts and aborts the upload when a part fails, so the role needs s3:AbortMultipartUpload. The policy does not allow s3:DeleteObject, so no run can delete a published archive.

The dagster-agent Terraform solution creates the bucket and deploys that CloudFormation template as a stack. Its apply must finish before the first materialization, including one on a branch deployment.

Versions​

The asset has one partition per year, starting at first_year. The partition key is January 1 of the year, in the format YYYY-MM-DD. The archive's tiles_version is the partition key. A year's partition exists from January 1 of that year.

A partition is built from Geofabrik's January 1 extract of its year, <region>-YY0101.osm.pbf. Geofabrik keeps the January 1 extract of every year since 2021, but it keeps the .md5 file only for the current year's extracts. The asset checks the download against the .md5 when Geofabrik has one, and otherwise against the Content-Length of a HEAD request. osm_extract_md5_checked in the materialization's metadata records which check ran.

Geofabrik can publish the January 1 extract as late as January 2, so a build on January 1 can fail with an HTTP 404.

Launching a build​

The osm_us_valhalla_tiles_schedule schedule builds the current year's partition at the time in cron_schedule, 06:00 UTC on January 3. Its default status on a branch deployment is stopped.

A build of the US runs longer than the 3-hour run_monitoring.max_runtime_seconds in dagster_cloud_settings.yaml. The osm_us_valhalla_tiles_job job sets the run tag dagster/max_runtime to max_runtime_hours, converted to seconds. The tag overrides that limit for the job's runs. Launch any other build, such as an earlier year or a test on a branch deployment, from that job. A run launched with Materialize on the asset does not have the tag.

Build​

  1. Download the extract and check its MD5, or its size when Geofabrik has no .md5 file for it. The asset retries 429 and 5xx responses and dropped connections. A retried download resumes from the last byte written.
  2. Build the tiles with pyvalhalla's valhalla_build_tiles, with one thread per CPU that os.process_cpu_count() reports in the step's task. The materialization records the count as build_threads. Valhalla writes its log to the step's stdout.
  3. Pack the tiles into the archive with valhalla_build_extract.
  4. Route a 5-minute auto isochrone from the point at check_latitude, check_longitude. The asset fails when the isochrone has no polygon.
  5. Upload the archive.

The build uses no admin, timezone, landmark, transit or traffic data. An auto isochrone without a departure time reads none of them. Without the admin database, Valhalla applies its defaults for country-specific access rules and for border crossings.

The tiles have no bicycle or pedestrian edges, so they support only the auto costing model.

Edge speeds come from OSM maxspeed tags and from Valhalla's default speed for each road class, so drive times are free-flow times.

Replacing a bad archive​

An archive that passes the isochrone check can still be wrong. No run can delete it. A new run of its partition records a materialization of the existing archive instead of building a new one. To rebuild a partition:

  1. Delete the object at its archive_uri with credentials that allow s3:DeleteObject on the bucket, such as an administrator's:
    aws s3 rm s3://ai-app-public-artifacts-prod/osm/us/valhalla/tiles/<tiles_version>/valhalla_tiles.tar
  2. Launch the partition from osm_us_valhalla_tiles_job.

Task size​

The task asks for:

  • 8 vCPU;
  • 60 GB of memory;
  • 100 GiB of ephemeral storage, set with op_tags: {ecs/ephemeral_storage: "100"}.

The US extract is about 12 GB and its archive about 15 GB. A build's working directory uses at most about 80 GiB of disk, during valhalla_build_tiles. The task asks for more than that because Fargate's ephemeral storage also holds the container image.