akanjilal.dev
Back to reference architectures
Reference architectureAmazon Web ServicesSelf-hosted CI build plane

Managed GitHub Actions runners on ECS

Continuous integration is where the source, the build environment, and the produced artifact all meet, which makes it the part of delivery a regulated organization most wants under its own control. This reference architecture runs GitHub Actions jobs on ephemeral AWS Fargate tasks inside a private virtual private cloud, builds container images without a Docker daemon, signs them inside the account, and pushes them to Amazon Elastic Container Registry, with the registry locked so that the runner is the only identity permitted to push. Each task runs one job, produces one artifact, and then exits.

Most teams adopt self-hosted runners for capacity or for access to a private network, and they run them long lived, shared, and broadly privileged. A regulated build plane needs the reverse. The design here treats the runner as disposable and the registry as a gate, because the control that actually matters is not where a build runs but what is allowed to land in the registry the platform deploys from. Everything else, including the ephemerality, the private networking, and the in-account signing, exists to make that guarantee true and provable.

Requirements

The requirements the build plane has to satisfy, the qualities it is judged on, and the constraints it must respect. Every decision in the next section traces back to one or more of them.

Functional requirements

NumberRequirement
1Run GitHub Actions jobs on self-hosted runners that live on private, account-owned compute.
2Build container images and push them to Amazon ECR.
3Provision capacity on demand for each queued job and reclaim it automatically when the job ends.
4Sign every image in the account and make only signed, runner-produced images deployable.

Quality attributes

NumberAttributeIntent
1EnforcementIt must be infeasible to land an artifact in the deployment registry without going through the runner.
2No public exposureEvery hop, from ingress to build to registry to signing, stays inside one account over private links, with no dependency on a public service.
3IsolationOne job cannot observe or tamper with another. No credential or filesystem is reused across jobs.
4Least privilegeEvery identity, the runner, the control plane, the registry, holds only the permissions it needs.
5No long-lived secretsNo personal access tokens, no static registry credentials, no reusable runner registration tokens.
6Observability and costEvery job is traceable, and parallelism and spend are bounded by design.

Constraints and assumptions

NumberConstraint or assumption
1An existing VPC with private subnets is available, with no route to an internet gateway or a network address translation gateway.
2For a genuinely private deployment the event source is GitHub Enterprise Server running inside the VPC, or GitHub Enterprise Cloud reached over PrivateLink.
3Fargate gives no Docker daemon and forbids privileged containers, so builds must be daemonless.
4Deployment targets can be configured to pull only from this registry and to verify a signature before admitting an image.

Architecture decisions and rationale

These are the decisions that shaped the build plane. Each records what was chosen, what was set aside, why, and what it costs.

1. Ephemeral single-use runners rather than a long-lived pool

Decision. Each queued job starts one fresh Fargate task that registers with a single-use just-in-time configuration, takes exactly one job, and exits.

Alternatives. A warm pool of persistent runners that pick up jobs over time.

Rationale. A runner that is reused is a runner that can carry one job's secrets, files, or compromise into the next, and a persistent runner is a standing target. A single-use task removes cross-job leakage entirely and makes scale-in a side effect of the job ending rather than a controller to operate. This serves the isolation attribute and functional requirement 3.

Consequences. Each job pays a task start latency of tens of seconds. A small warm pool can front the queue where that latency is unacceptable, trading a little isolation purity for speed.

2. ECS Fargate with a daemonless builder rather than a managed Docker host

Decision. Run the runner on Fargate and build images with rootless BuildKit, with kaniko as a documented alternative.

Alternatives. ECS on EC2 or self-managed instances with the Docker daemon, allowing classic Docker-in-Docker builds.

Rationale. Fargate is managed in the truest sense: there is no host to patch, no daemon to secure, and each task is its own kernel-isolated micro virtual machine. The cost of that is no Docker daemon and no privileged mode, which a daemonless, rootless builder resolves by producing an image and pushing it straight to the registry without a socket. This serves the no-public and least-privilege attributes by removing the host as an attack surface.

Consequences. A team that genuinely needs full Docker compatibility uses the same control plane against an ECS-on-EC2 capacity provider and accepts host patching and weaker isolation in exchange.

3. A queue between the webhook and the build rather than a direct call

Decision. The webhook receiver verifies and enqueues to Amazon SQS, and a separate scale-up function drains the queue and launches tasks, with a dead-letter queue for poison messages.

Alternatives. Launching a task directly from the webhook handler.

Rationale. The queue decouples ingestion from provisioning, so bursts, GitHub's at-least-once webhook redeliveries, and transient AWS errors are absorbed and retried rather than dropped, and reserved concurrency on the scale-up function becomes a hard ceiling on simultaneous launches. This serves the observability and cost attribute and reliability.

Consequences. A duplicate webhook can launch one extra runner, which is harmless because an idle ephemeral runner simply times out.

4. A GitHub App with just-in-time registration rather than tokens

Decision. Authenticate to GitHub as a GitHub App, mint a short-lived installation token, and request a single-use just-in-time runner configuration per task.

Alternatives. A personal access token, or a reusable runner registration token baked into the image.

Rationale. App installation tokens are short lived and scoped, and a just-in-time configuration cannot be reused once its runner comes online, so there is no durable registration secret to steal. The app private key is the only long-lived credential, and it lives in Secrets Manager. This serves the no-long-lived-secrets attribute.

Consequences. The control plane carries a small amount of GitHub App plumbing, which is the accepted cost of holding no reusable tokens.

5. The registry policy as the enforcement point rather than process or trust

Decision. The ECR repository policy allows the push actions only to the runner task role and explicitly denies every other principal, and deploy targets pull only from this registry.

Alternatives. Relying on workflow conventions, branch protection, or reviewer discipline to keep builds on the runner.

Rationale. Enforcement is the requirement most designs address only loosely. Governance in GitHub makes building elsewhere hard to introduce, but only the registry policy makes it impossible to land elsewhere, because even an image built on a laptop cannot be pushed. This serves the enforcement attribute and functional requirement 2.

Consequences. The runner task role becomes a sensitive identity whose scope must be kept narrow, which the design does by granting it push to named repositories only.

6. In-account signing with a KMS key rather than keyless Sigstore

Decision. Sign every image with cosign backed by an asymmetric AWS KMS key, with the transparency log upload disabled.

Alternatives. Keyless cosign signing against the public Sigstore certificate authority and transparency log.

Rationale. A KMS-backed signature is not keyless, so there is no call to a public certificate authority, and disabling the transparency log upload means no write to a public log either. The only services touched are KMS and ECR, both of which have a private endpoint, so signing keeps the no-public guarantee. This serves the no-public-exposure attribute and functional requirement 4.

Consequences. Verification distributes the KMS public key rather than relying on a public trust root, which is a deliberate trade for staying inside the account. AWS Signer with Notation is an equivalent managed alternative.

7. Every dependency private rather than reached over the internet

Decision. Put the runner and the control-plane functions in private subnets with egress restricted to the VPC, reach every AWS service over an account-scoped VPC endpoint, ingest the webhook through a private REST API, and source base images through an ECR pull-through cache.

Alternatives. A NAT gateway for outbound access and public service endpoints.

Rationale. Each external dependency a naive design would reach over the internet has a private replacement here, which removes the exfiltration paths and the dependence on public services in one move, and the absence of a NAT gateway also removes its data-processing charge. This serves the no-public-exposure attribute.

Consequences. The one unavoidable caveat is that GitHub.com cloud lives on the public internet, so a genuinely private deployment uses GitHub Enterprise Server inside the VPC.

The build lifecycle

Every build follows the same path, and the value of the design is that there is only one. A queued job becomes a single ephemeral runner that registers, takes exactly one job, builds, pushes, signs, verifies, and exits. None of these steps can be skipped, because the job has no other runner to land on.

The eight steps of one build: verify and enqueue, mint a just-in-time config, RunTask on Fargate, register and long-poll for one job, build with rootless BuildKit, push to ECR, sign and attest with cosign and KMS, then verify and exit, producing a signed image that deploy targets admit.
One build, eight steps. The runner registers with a single-use configuration, takes one job, and deregisters, so scale-in is simply the task exiting.

The architecture

A control plane decides when and how many runners to start, and a data plane does the build. The webhook receiver is a deliberately tiny, internet-facing function that verifies the signature, ignores anything that is not a queued job targeting the runner labels, and enqueues a compact message. The scale-up function drains the queue, mints the just-in-time configuration, and launches the task. Everything else, Secrets Manager, KMS, the VPC endpoints, and the alarms, supports those two paths.

GitHub Enterprise Server in the VPC posts a workflow_job webhook to a private API Gateway, a webhook Lambda verifies and enqueues to SQS with a dead-letter queue, a scale-up Lambda mints a just-in-time config and runs one Fargate task that builds and pushes a signed image to ECR, which deploy targets pull and verify. Secrets Manager, KMS, account-scoped VPC endpoints, and CloudWatch support the flow.
The control plane, from the private webhook to the scale-up function, and the data plane, the single Fargate task. They are decoupled by the queue so bursts and redeliveries are absorbed rather than dropped.

The runner image bundles the GitHub Actions runner, the Amazon ECR credential helper, rootless BuildKit, and the supply-chain tools. Its entrypoint configures the credential helper for the task's account and region, starts the build engine without a daemon and without privilege, and runs the runner with the just-in-time configuration. Registry authentication is implicit through the task role, so there are no registry credentials anywhere in the pipeline.

Networking

The runner and the control-plane functions sit in private subnets with no route to an internet gateway or a network address translation gateway. Their security groups permit no inbound traffic and egress only to the VPC, so they can reach the VPC endpoints and the in-VPC GitHub Enterprise Server and nothing on the internet. Every AWS service is consumed over an interface or gateway endpoint whose policy pins usage to this account, and the webhook ingress is a private REST API that resolves only through the execute-api endpoint.

Inside one private VPC with no internet gateway or NAT, GitHub Enterprise Server, the control-plane Lambdas, and the Fargate runner reach AWS only through account-scoped VPC endpoints: execute-api, ecr.api and ecr.dkr, kms, secretsmanager, sts, sqs, ecs and logs, and an S3 gateway endpoint. The public internet has no route in or out.
Every dependency private, scoped to one account. The KMS endpoint is what lets in-account signing happen without leaving the VPC, and the public internet has no route to the runner or the functions.
Would-be public dependencyPrivate replacement
GitHub.com webhook and runner long-pollGitHub Enterprise Server inside the VPC, on RFC1918 addresses
Public API Gateway ingressA private REST API behind the execute-api endpoint
ECR, S3, Secrets, STS, KMS, SQS, ECS over the internetAccount-scoped VPC endpoints
Docker Hub or public registry base imagesA private ECR pull-through cache
Keyless cosign against public Sigstorecosign with an in-account KMS key, transparency log disabled
NAT gateway to the internetRemoved entirely

Enforcing private builds only

Enforcement works as defense in depth across three layers, and the third is the one that makes it stick. Source governance in GitHub, code owners on the workflow files, an allowed-actions policy, and a pinned-commit ruleset, makes building elsewhere hard to introduce. Capacity gating means a job only receives a runner if it requests the exact labels, so a misconfigured job simply never gets capacity rather than falling back to a hosted runner. The registry gate is the real control.

The ECR app repository policy allows PutImage only from the runner task role and denies it for a laptop, a rogue CI pipeline, and any other principal. Deploy targets pull only from the repository and verify the cosign signature before admitting an image.
The registry gate. Only the runner task role can push, so even an image built elsewhere cannot reach the repository the platform deploys from.

The repository policy grants the push actions to the runner task role and denies them to everyone else, which is what makes the claim that all artifacts are built privately actually true rather than aspirational.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "OnlyRunnerCanPush",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<account>:role/gha-ecs-runner-task" },
      "Action": [
        "ecr:PutImage",
        "ecr:InitiateLayerUpload",
        "ecr:UploadLayerPart",
        "ecr:CompleteLayerUpload",
        "ecr:BatchCheckLayerAvailability"
      ]
    },
    {
      "Sid": "DenyPushFromAnyoneElse",
      "Effect": "Deny",
      "Principal": "*",
      "Action": [
        "ecr:PutImage",
        "ecr:InitiateLayerUpload",
        "ecr:UploadLayerPart",
        "ecr:CompleteLayerUpload"
      ],
      "Condition": {
        "ArnNotEquals": {
          "aws:PrincipalArn": "arn:aws:iam::<account>:role/gha-ecs-runner-task"
        }
      }
    }
  ]
}

In-account signing

Signing is a first-class control here, and it is done entirely inside the account with no public dependency. The build pushes the image, cosign signs it with an in-account KMS key and the transparency log disabled, and the signature and a software bill of materials attestation are stored as artifacts in ECR alongside the image. The build verifies its own signature before treating the artifact as releasable, and deploy targets run the same verification with the key's public half before admitting an image.

After the build pushes to ECR, cosign signs with an in-account AWS KMS key and the transparency log disabled, so there is no call to public Sigstore Fulcio or Rekor. The signature and a syft SBOM attestation are stored in ECR, and deploy targets verify with the KMS public key before admitting the image.
Signed inside the account. Because the key is in KMS and the transparency log is disabled, signing touches only KMS and ECR, both over private links, with no call to public Sigstore.
# Sign with the in-account KMS key. --tlog-upload=false means no call to the
# public Sigstore transparency log: signing touches only KMS and ECR.
cosign sign   --yes --tlog-upload=false --key "$SIGNING_KMS_URI" "$IMAGE"
cosign attest --yes --tlog-upload=false --key "$SIGNING_KMS_URI" \
  --predicate sbom.spdx.json --type spdxjson "$IMAGE"

# Verify in-pipeline and again at deploy time, against the same in-account key.
cosign verify --insecure-ignore-tlog=true --key "$SIGNING_KMS_URI" "$IMAGE"

Combined with the registry gate, the deployable set is exactly the images built by the runner and signed by the in-account key. An admission controller or a deploy gate configured with the KMS public key is what turns that into a rule the platform enforces on every release.

The implementation, the Terraform, the control-plane functions, the runner image, and the sample workflow, is open source, and a guided walkthrough proves the three claims, that the build is private, that only the runner can push, and that every image is signed.

References

  1. GitHub, Autoscaling with self-hosted runners and just-in-time runners.https://docs.github.com/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners
  2. GitHub, Authenticating as a GitHub App installation.https://docs.github.com/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation
  3. Amazon Web Services, Amazon ECS on AWS Fargate.https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html
  4. Amazon Web Services, Amazon ECR private repository policies.https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-policies.html
  5. Amazon Web Services, AWS PrivateLink and VPC endpoints.https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html
  6. The Moby Project, rootless BuildKit.https://github.com/moby/buildkit/blob/master/docs/rootless.md
  7. Sigstore, cosign signing with an AWS KMS key.https://docs.sigstore.dev/cosign/key_management/signing_with_kms/
  8. Amazon Web Services, Amazon Inspector enhanced scanning for Amazon ECR.https://docs.aws.amazon.com/inspector/latest/user/scanning-ecr.html