Skip to main content
Monitoring changes ship through the Better Stack Terraform GitHub Actions workflow (.github/workflows/betterstack-terraform.yml). It has two clearly separated lanes: a static validate lane that runs automatically on pull requests, and a plan/apply lane that only runs when a human dispatches it.

​
Two lanes at a glance

​
Validate lane — automatic, safe by construction

Triggered on every pull request to master, release or develop that touches infra/betterstack/**, the runner script, or the workflow file itself. It runs with no secrets and no state, so it physically cannot change Better Stack. This is the lane you see turn green on a monitoring PR. It proves the config is well-formed; it does not talk to Better Stack.

​
Plan/apply lane — manual, gated, per environment

This lane runs via workflow_dispatch (Actions tab → Run workflow) or as the final stage of the Release Train orchestrator (.github/workflows/release-train.yml). It takes two inputs — the target environment (a fixed choice list: development, certification, production) and the action (plan or apply) — plus a confirmation string. Dispatch runs are tag-only: pick a CalVer release tag in “Use workflow from”. The shared guard (.github/scripts/validate-release-tag.sh) rejects branch refs and enforces the channel matrix (alpha → development, rc → development + certification, stable → all three), and requires the tag to have a GitHub Release.

​
The confirmation gate

The gate job runs first. If action = apply and the confirm_apply input is not exactly APPLY, it fails before anything touches infrastructure. plan needs no confirmation — it is read-only. When the workflow is called by the Release Train, the train’s single RELEASE confirmation replaces the typed APPLY (via a workflow_call-only confirmed input that manual dispatches cannot set). The gate then validates the release tag with the shared guard.

​
Environment isolation and secrets

The terraform job runs inside the selected GitHub Environment, so that environment’s protection rules (required reviewers, wait timers) and its scoped secrets apply automatically.

Per-environment secrets

BETTERSTACK_UPTIME_API_TOKEN, HEALTH_MONITOR_TOKEN, and optional LOGTAIL_API_TOKEN. Defined on each of development, certification and production.

Repository secrets (remote state)

TF_BACKEND_OVERRIDE (the full backend "s3" block), TF_STATE_ACCESS_KEY and TF_STATE_SECRET_KEY. Shared across environments; state stays isolated by workspace.

​
Fail-fast on missing remote state

Before init, the job checks that TF_BACKEND_OVERRIDE exists. If it does not, it errors out immediately — CI must never plan against an empty local state, which would try to recreate every resource. When present, the block is written to backend_override.tf and terraform init picks up the remote backend.

​
Concurrency lock

A concurrency group keyed by environment (betterstack-terraform-<environment>) with cancel-in-progress: false guarantees two operations never run against the same environment at once; a second run queues behind the first.

​
Outputs

On apply, the final step appends terraform output to the job summary so the resulting status_page_resources_json and uptime_heartbeat_ids are visible without re-running anything.

​
How to run it

1

Open the workflow

GitHub → Actions → Better Stack Terraform → Run workflow, and pick the release tag in “Use workflow from” (branch refs are rejected).
2

Run a plan first

Pick the target environment, set action = plan, and run. Read the diff in the job log. A 0 to add, 0 to change, 0 to destroy plan means no drift.
3

Apply when the plan is what you expect

Run again with action = apply and type APPLY in confirm_apply. The gate passes, the job runs inside the environment, and the outputs summary is posted.
4

Confirm convergence

Run one more plan — it should report no changes.

​
Safety decisions

Monitoring changes affect the public status page and can page on-call engineers. A human must read the plan and explicitly type APPLY, so an infrastructure change is always a deliberate, reviewed action — never a side effect of merging.
Terraform tracks resource ids in its state. Running against empty local state would make Terraform believe nothing exists and recreate every monitor, duplicating resources on the status page. Requiring TF_BACKEND_OVERRIDE makes that failure impossible.
Two overlapping applies against the same state can corrupt it or race on the Better Stack API. The per-environment lock serializes operations while still allowing different environments to run in parallel.
PR validation should be safe to run on any change, including from forks. By using init -backend=false and no tokens, the lane can check correctness without any ability to reach or mutate Better Stack.
  • Workflow: .github/workflows/betterstack-terraform.yml
  • Runner: scripts/status-monitoring/terraform-apply.sh
  • Backend template: infra/betterstack/backend_override.tf.example