Zero trust AI gateway

The phrase zero trust means that nothing is trusted because of where it sits. A request from inside the network is treated with the same suspicion as a request from outside, every call is checked on its own merits, and the gateway grants the least it can rather than the most it could. For systems built on models and agents this matters more than usual, because an agent is, by design, a component that decides at run time what to call and with what arguments, so the controls cannot live only in the application. They have to live on the path that every call must cross.

Requirements

The requirements the gateway has to satisfy, the quality attributes it is judged on, the constraints it must respect, and the assumptions it rests on. Every decision in the next section traces back to one or more of them.

Functional requirements

Number	Requirement
1	Provide one controlled entry point between callers and all models, tools, and knowledge.
2	Authenticate every caller and authorize each model and tool call against an allow list.
3	Inspect every prompt and every response for prompt attacks and for personal data.
4	Issue short lived, scoped credentials to tools rather than sharing standing secrets.
5	Require a human to approve any action that moves money or is otherwise irreversible.
6	Record every model invocation and every tool call as an audit trail.
7	Continue to enforce all of the above after a failover to the second region.

Quality attributes

Number	Attribute	Intent
1	Security	No caller reaches a model or a tool except through the gateway, with least privilege throughout.
2	Auditability	Reconstruct every interaction after the fact, retained for at least one year.
3	Reliability	Tolerate the loss of an availability zone with no action, and the loss of a region within the recovery objective.
4	Latency	Add only a small and bounded overhead to each call.
5	Extensibility	Add a model or a tool by registering it, without changing any caller.
6	Cost efficiency	Pay for inspection and routing in proportion to use.

Constraints and assumptions

Number	Constraint
1	All inference and all data remain within Canada and within the two approved regions.
2	The gateway sits inside the landing zone production segment and inherits its inspection and guardrails.
3	Only approved foundation models and approved tool versions may be reached.
4	Personal data and cardholder data must never leave the private network.

Number	Assumption
1	Callers are internal applications and agents that already authenticate to the corporate identity provider.
2	Tools expose a described interface that can be registered and validated.
3	Foundation models are consumed through Amazon Bedrock rather than self hosted.

Architecture decisions and rationale

These are the decisions that shaped the gateway. Each records what was chosen, what was set aside, why, and what it costs.

1. One gateway chokepoint rather than each application calling models directly

Decision. Route every call to a model, a tool, or a knowledge source through a single gateway, and forbid direct calls.

Alternatives. Letting each application hold its own credentials and call Amazon Bedrock and the tools directly, with controls embedded in each application.

Rationale. A single path is the only place a control can be proven to apply to every call, where controls spread across many applications are applied inconsistently and are impossible to audit as a whole. This serves functional requirements 1 and 6 and the security attribute.

Consequences. The gateway is on the critical path for every call, so it is built across two availability zones and two regions and is kept thin enough to add only a small latency.

2. Bedrock AgentCore for the runtime, identity, and tool gateway rather than building the plumbing

Decision. Use Bedrock AgentCore for the agent runtime with session isolation, for scoped identity, and for the tool gateway.

Alternatives. Assembling the same capabilities from containers, a custom token service, and hand written tool adapters.

Rationale. The managed runtime provides session isolation, a tool gateway, identity, and observability as building blocks, which removes a large amount of undifferentiated and security sensitive code that would otherwise have to be written and maintained. This serves the extensibility attribute and functional requirement 4.

Consequences. The design is tied to a managed service, which is an accepted trade because the alternative is to maintain security critical plumbing that is not the organization's core business.

3. Bedrock Guardrails applied independently rather than relying on a model's built in safety

Decision. Apply Bedrock Guardrails to both the input and the output of every call using the independent guardrail interface, so the same policy holds for any model.

Alternatives. Trusting the safety behavior built into each model, or filtering only the input.

Rationale. A guardrail applied independently enforces one consistent policy across every model and is the place to catch prompt attacks on the way in and personal data or secrets on the way out, which a single model's behavior cannot be relied upon to do. This serves functional requirement 3 and the security attribute.

Consequences. Each call carries two inspection steps, which is a small and bounded cost accepted for a consistent control.

4. An allow list of models and tools rather than open access

Decision. Maintain a registry of approved models and tools, pin their versions, and reject anything not on the list.

Alternatives. Allowing callers to reach any model or any reachable service.

Rationale. An allow list turns the question of what an agent may do into a reviewed decision rather than an emergent one, which is what a regulated environment requires. This serves functional requirement 2 and constraint 3.

Consequences. Adding a capability requires a registration step, which is the intended point of control.

5. Short lived, scoped credentials per tool rather than shared secrets

Decision. Issue a credential that is scoped to a single tool and a single purpose, valid for a short time, through AgentCore Identity.

Alternatives. A shared service credential that every tool call uses.

Rationale. A scoped and short lived credential limits what a compromised call can do and for how long, where a shared standing secret would expose every tool at once if it leaked. This serves functional requirement 4 and the security attribute.

Consequences. The identity service must mint credentials on demand, which is the managed behavior of AgentCore Identity.

6. A human approval step for high impact actions rather than full autonomy

Decision. Require a person to confirm any action that moves money, changes a record of value, or is otherwise irreversible.

Alternatives. Allowing the agent to complete such actions on its own once it decides to.

Rationale. An agent's judgment is probabilistic, so the cost of a wrong irreversible action is too high to leave unattended, and a single confirmation point removes the most damaging class of mistake. This serves functional requirement 5.

Consequences. Some actions are slower because they wait for a person, which is accepted in exchange for preventing the worst outcomes.

7. Two regions in warm standby rather than a single region

Decision. Run the gateway active in the primary region and warm in the secondary, with configuration and session state replicated and traffic directed by Amazon Route 53.

Alternatives. A single region, accepting that a regional failure takes the capability offline.

Rationale. The controls are only useful if they are available, so the gateway has to survive a regional failure within the recovery objective. This serves functional requirement 7 and the reliability attribute.

Consequences. A standby region carries a running cost, which is reduced by keeping it warm rather than fully scaled until it is needed.

The request path

Every call follows the same path, and the value of the design is that there is only one. A caller arrives, is authenticated and given a scope, the prompt is inspected, the requested model or tool is checked against the allow list, the call is made over a private path, the response is inspected, and the whole exchange is recorded. None of these steps can be skipped, because the caller has no other way to reach a model or a tool.

The inspection on the way in and the way out is done by Bedrock Guardrails, applied independently of the model so the policy is the same regardless of which model is called. A representative guardrail blocks prompt attacks on the input, blocks card numbers and the Canadian social insurance number, anonymizes contact details, and denies a set of topics the business has decided are out of bounds.

{
  "name": "payments-gateway-guardrail",
  "contentPolicyConfig": {
    "filtersConfig": [
      { "type": "PROMPT_ATTACK", "inputStrength": "HIGH", "outputStrength": "NONE" },
      { "type": "MISCONDUCT",    "inputStrength": "HIGH", "outputStrength": "HIGH" }
    ]
  },
  "sensitiveInformationPolicyConfig": {
    "piiEntitiesConfig": [
      { "type": "CREDIT_DEBIT_CARD_NUMBER",      "action": "BLOCK" },
      { "type": "CA_SOCIAL_INSURANCE_NUMBER",    "action": "BLOCK" },
      { "type": "EMAIL",                         "action": "ANONYMIZE" }
    ]
  },
  "topicPolicyConfig": {
    "topicsConfig": [
      { "name": "Investment advice", "type": "DENY",
        "definition": "Recommendations to buy or sell specific securities" }
    ]
  }
}

The deployment

The gateway runs inside the landing zone, in the production segment of the network, which means it already sits behind the centralized inspection and inherits the organization's identity and logging. In front of it are the web application firewall and the content delivery network, which absorb common attacks and rate limit abusive callers. The runtime, the identity service, and the guardrails run inside the production virtual private cloud, and the model service, the secrets, and the session store are reached over private endpoints so that no part of the path crosses the public internet.

The tool boundary

A tool call is the highest risk action an agent takes, because a tool acts on external systems. The gateway therefore applies its strictest controls at the boundary between the agent and the tools. Only tools in the registry can be called, each call is made with a credential scoped to that one tool, the arguments are validated against the tool's declared interface before the call is made, an action that is irreversible waits for a human, and every call is recorded. Third party tools are treated as untrusted by default, which means their responses are inspected on the way back just as a user's input is.

This is the same discipline a payment network applies to its own integrations, carried over to agents. The agent may propose any action, and the gateway ensures that the action it actually performs is permitted, scoped, validated, and recorded. That separation is what makes it acceptable to place an agent in front of production systems.