# Handoff: implement the five-tier IPsec ingress architecture

## What this is

A handoff from the **`ip-sec-tf-prototype`** Terraform prototype into this
Pulumi repo. The prototype validated end-to-end TCP flow over IPsec →
internal NLB → ECS Fargate (running `frontdoor-mtls`) → zrok / OpenZiti →
customer backend service. That pattern is now ready to be expressed as
proper per-tier Pulumi stacks here, where it belongs long-term.

Read this doc end-to-end before making code changes in response. Then cross-
reference the canonical sources below before designing anything; they're the
source of truth.

## Canonical sources (read these, don't re-derive)

| Source | What's there |
|---|---|
| Repo: [`netfoundry/ip-sec-tf-prototype`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype) | The working prototype. Reference implementation for every resource this project will create. Local path: `../ip-sec-tf-prototype/`. |
| `../ip-sec-tf-prototype/ARCHITECTURE.md` | The approved five-tier classification (Platform / Hosting Region / Tenant / Endpoint Site / Share) with every cloud resource mapped. **Authoritative.** Read this first. |
| `../ip-sec-tf-prototype/diagrams/tiers.png` (source: `tiers.dot`) | The tier diagram. Arrows point downward: "upper tier provides foundations for lower tier." |
| Confluence [`ipSec Proposed Architecture`](https://netfoundry.atlassian.net/wiki/spaces/OSS/pages/4996169734/ipSec+Proposed+Architecture) | Wider-audience copy of ARCHITECTURE.md. Always kept in sync with the repo markdown and diagram — this is the "living triumvirate." |
| `../ip-sec-tf-prototype/TESTING.md` | Validation procedures, including the zrok-loopback self-validation trick. Copy the pattern into this project's test harness when implementing. |
| `../ip-sec-tf-prototype/PLAN-reserved-share-tokens.md` | Open work on the backend-side zrok shares (reserved tokens + systemd). Share stack should eventually drive this. |
| Existing template: [`templates/frontdoor-tcp-shares/`](./templates/frontdoor-tcp-shares/) | The pattern the prototype mirrored. The Share tier maps almost 1:1 to this. |
| `../frontdoor-mtls/ARCHITECTURE.md` and `CHANGE-support-disabled-tls.md` | Recent app-side changes that unblocked the prototype (TLS-disabled support, v2 clientVersionCheck handling). Share stack must consume an image built after those land (`v2-494b642-0846c86` or newer as of this writing). |
| `../frontdoor-mtls-cloud/Dockerfile.v2` + `frontdoor_mtls_entrypoint.py` | The container the Share tier runs. Entrypoint env-var contract documented there. |

## What the prototype proved (so you don't have to re-prove it)

- **Pinned private NLB IP** via `subnet_mapping.private_ipv4_address` works,
  and is required for the customer-side firewall-whitelist story.
- **`preserve_client_ip=true`** on target groups means task-side SG rules
  must be CIDR-based (endpoint CIDR + VPC CIDR), not SG-referenced.
- **Cross-account ECR pull** from `745435643501` to workload accounts is
  live and working.
- **`FRONTDOOR_TLS_ENABLED=false`** path is supported by `frontdoor-mtls`
  as of the v2-494b642-0846c86 image (IPsec provides the confidentiality,
  no need for mTLS at this phase).
- **Zrok v2 `clientVersionCheck` 202** handling is fixed in the same image.
- **zrok fabric loopback** via `zrok2 access private` on the backend host
  is a powerful debugging tool — use it to isolate AWS-side vs zrok-side
  failures. Replicate in this project's test harness.
- **Network primitives are fully modeled.** The prototype treats VPC,
  IGW, subnets, VPC route tables, TGW, TGW VPC attachment, TGW route
  tables, Customer Gateway, and VPN Connection as first-class Terraform
  resources (not `data` lookups). That means the HCL is a faithful
  description of every attribute the Pulumi templates need to cover —
  start from the reference files below when scaffolding.

## Reference implementation: the live network as HCL

The prototype's tier-organised Terraform code is the closest thing to a
spec for your Pulumi templates. Translate these files into Pulumi Go and
you've covered 90% of the Hosting Region + Endpoint Site tiers:

| File | Tier | What's in it |
|---|---|---|
| [`../ip-sec-tf-prototype/hosting_region.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/hosting_region.tf) | Hosting Region (shared) | VPC, IGW, 4 subnets, 4 VPC route tables + 3 explicit associations, TGW, TGW VPC attachment (`hub_vpc_attachment1`), `ZT_HUB` TGW route table, VPC → ZT_HUB association + propagation |
| [`../ip-sec-tf-prototype/endpoint_cust_a.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/endpoint_cust_a.tf) | Endpoint Site (per-IPsec) | Customer Gateway, VPN Connection (TGW-attached), `Customer_A` TGW route table + associations + propagations, 3 static TGW routes (2 × `172.31.0.0/16` → VPN attachment, 1 × `100.64.0.0/10` → VPC attachment) |
| [`../ip-sec-tf-prototype/security_groups.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/security_groups.tf) | Endpoint Site (NLB SG, tcp_share SG) | `preserve_client_ip=true`-aware CIDR ingress |
| [`../ip-sec-tf-prototype/nlb.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/nlb.tf) | Endpoint Site (NLB) + Share (TG + listener) | Pinned IP via `subnet_mapping` |
| [`../ip-sec-tf-prototype/ecs.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/ecs.tf) | Hosting Region (cluster) + Share (task def, service) | Env-var contract for `frontdoor-mtls` |

### Inputs vs. platform conventions

The prototype pinned specific IDs, CIDRs, and names because it was a
single-tenant throwaway. The Pulumi templates won't reuse those IDs —
every Hosting Region and Endpoint Site will get fresh resources. What
matters here is which of these values are **user-supplied at stack
creation** vs. **fixed by platform convention**.

**Hosting Region stack — per AWS account + region + VPC:**

| Input or constant | Nature | Notes |
|---|---|---|
| Region label / stack name | User input | Drives naming prefixes and stack identity. |
| AWS region | User input | e.g. `us-east-1`. |
| VPC CIDR | User input | Must not overlap with any expected endpoint CIDR. |
| AZs to use | User input | Today's prototype uses one (`us-east-1d`); production likely wants ≥2. |
| IPsec subnet CIDR per AZ | User input | This subnet is where the TGW VPC attachment is enrolled and where Fargate tasks land. `/24` is plenty for the prototype; size per expected task count. |
| `tags` schema | Platform constant | `Project` + `Owner` + `Environment` (+ `Tenant` on tenant-scoped resources). Apply via provider `default_tags`. |
| `aws.ec2.TransitGateway.AmazonSideAsn` | Platform input, mostly stable | Per-TGW. Pick from the private ASN range. |
| `aws.ec2.TransitGateway.AutoAcceptSharedAttachments` | Platform constant | `disable`. We don't cross-account-share TGWs in the current model. |
| `DefaultRouteTableAssociation` / `DefaultRouteTablePropagation` | Platform constant | Both `disable`. Association + propagation are always wired explicitly, never defaulted. Mirroring this in Pulumi is required or you'll get a routing soup. |
| `aws.ec2.TransitGateway.DnsSupport` | Platform constant | `enable`. |
| `aws.ec2.TransitGateway.VpnEcmpSupport` | Platform constant | `disable`. (Provider default is `enable`; override explicitly.) |
| `aws.ec2.TransitGatewayVpcAttachment.DnsSupport` | Platform constant | `enable`. |
| `aws.ec2.TransitGatewayVpcAttachment.SecurityGroupReferencingSupport` | Platform constant | `enable`. |
| `aws.ec2.TransitGatewayVpcAttachment.Ipv6Support` / `ApplianceModeSupport` | Platform constants | Both `disable` until we actually need them. |
| `aws.ec2.TransitGatewayVpcAttachment.TransitGatewayDefaultRouteTable{Association,Propagation}` | Platform constants | Both `false`. We explicitly associate with the shared VPC-side TGW RT (e.g. `ZT_HUB`-equivalent). |
| Shared TGW-VPC RT name | Platform convention | One per Hosting Region, shared across every Endpoint Site whose CIDRs don't conflict. Name it something like `<region>-hub-rt`. |

**Endpoint Site stack — per IPsec ingress:**

| Input or constant | Nature | Notes |
|---|---|---|
| Endpoint name/label | User input | Used for resource naming + tags. e.g. `cust-a`, `acme-hq`. |
| Customer-peer public IP (`aws.ec2.CustomerGateway.IpAddress`) | User input | The on-prem IPsec device's public IP. |
| Customer-peer BGP ASN (`aws.ec2.CustomerGateway.BgpAsn`) | User input | From the endpoint's config. Irrelevant when we use static routes only, but still required by AWS. |
| Endpoint CIDR | User input | The on-prem network that traffic originates from. **The single most important input** — drives the 3 static routes, SG ingress rules, and TGW propagation scope. |
| Tunnel inside CIDRs (`Tunnel1InsideCidr`, `Tunnel2InsideCidr`) | User input, optional | Pin for stability across regenerations; let AWS pick if you don't care. Prototype pins `169.254.69.116/30` + `169.254.25.148/30`. |
| PSKs | User input, optional | AWS-generated if omitted. Pin when the customer side needs a known value. Store in Pulumi Secrets, not stack config. |
| `aws.ec2.CustomerGateway.Type` | Platform constant | `ipsec.1`. |
| `aws.ec2.VpnConnection.StaticRoutesOnly` | Platform constant | `true`. We drive routing from the TGW side, not BGP. |
| `aws.ec2.VpnConnection.LocalIpv4NetworkCidr` | Platform constant | `0.0.0.0/0` — AWS side accepts any destination. |
| `aws.ec2.VpnConnection.RemoteIpv4NetworkCidr` | Platform convention | Set to the *Hosting Region's IPsec subnet CIDR* (not the endpoint CIDR). This is the AWS-side traffic selector. |
| `aws.ec2.VpnConnection.OutsideIpAddressType` | Platform constant | `PublicIpv4`. |
| `aws.ec2.VpnConnection.PresharedKeyStorage` | Platform constant | `Standard` (Secrets-Manager-backed option exists; revisit when PSK rotation is productised). |
| `aws.ec2.VpnConnection.EnableAcceleration` | Platform constant | `false`. Turn on if/when we measure actual latency benefit. |
| `aws.ec2.VpnConnection.TunnelInsideIpVersion` | Platform constant | `ipv4`. |
| Per-endpoint TGW RT name | Platform convention | One per endpoint. Name like `<endpoint-name>-rt` so a regex cleanup or audit is straightforward. |
| VPN-attachment → per-endpoint-RT association | Platform constant | Always exactly one. |
| VPC-attachment → per-endpoint-RT propagation | Platform constant | Always enabled — lets the VPN side know about VPC destinations. |
| VPN-attachment → shared-hub-RT propagation | Platform constant | Always enabled — symmetric. |

**Share stack — per published TCP app** (see the existing
`templates/frontdoor-tcp-shares/` for parameters; the prototype's
`ecs.tf` + `nlb.tf` + `security_groups.tf` document the shape):

| Input or constant | Nature | Notes |
|---|---|---|
| Share token | User input | The zrok tcpShare token. |
| Tenant's zrok account enable token | User input (tenant-scoped, inherited) | `FRONTDOOR_ENABLE_TOKEN`. |
| Listener port on the Endpoint's NLB | User input | 9001/9002/9003 in the prototype; production may want a reserved range with lookup. |
| `frontdoor-mtls` image tag | Platform input | Pinned at the stack level; bump per release. |
| Container bind port | Platform constant | `9191` (zrok-orchestration convention). |
| Target group health-check params | Platform constants | `protocol=TCP`, `interval=30`, `timeout=15`, `healthy=2`, `unhealthy=4`, `deregistration_delay=30`. |
| `aws.lb.TargetGroup.PreserveClientIp` | Platform constant | `true`. Drives the CIDR-based SG rule shape. |
| ECS service deployment config | Platform constants | `DeploymentCircuitBreaker{enable=true, rollback=true}`, `ForceNewDeployment=true`, `PropagateTags=SERVICE`, `EnableEcsManagedTags=true`, `HealthCheckGracePeriodSeconds=60`. |
| Task sizing | Platform defaults, user override OK | `cpu=512`, `memory=1024`. |
| `FRONTDOOR_TLS_ENABLED` | User input (bool, per share) | `false` today (IPsec path is confidential). Production will want `true` with per-share client CA + server cert material. |

### Resources that are out-of-scope everywhere

These exist in the prototype environment but aren't part of the tier
model and shouldn't be created by the Pulumi templates:

- **NetFoundry-edge VPN router EC2 + its ENI** — an unrelated ingress
  path, left in place so NetEng can still use it. The prototype's
  `aws_route_table.ziti_isolation` references this ENI by ID in a
  `100.64.0.0/10` route; if we keep that NetFoundry-edge path in the
  managed Endpoint Site template, it goes in as a user-supplied ENI ID
  (or skipped entirely for tenants that don't need it).
- **Any "Regression-*" VPC endpoints** — belong to other projects.

### NetEng's 3-static-routes pattern — bake this into the Endpoint-Site Pulumi template

Per the NetEng team, every Endpoint Site onboarding must add **3 static
routes** that all point at the customer's on-prem CIDR. These are the
per-endpoint sprinkles on the shared Hosting-Region plumbing:

| # | Where | Destination | Target |
|---|---|---|---|
| 1 | VPC subnet RT attached to the IPsec subnet | `<endpoint-CIDR>` (e.g. `172.31.0.0/16`) | TGW |
| 2 | Shared TGW VPC RT (`ZT_HUB`) | `<endpoint-CIDR>` | This endpoint's VPN attachment |
| 3 | Per-endpoint TGW VPN RT (`Customer_A`) | `<endpoint-CIDR>` | Same VPN attachment (loopback-style) |

All 3 are created in
[`../ip-sec-tf-prototype/endpoint_cust_a.tf`](https://bitbucket.org/netfoundry/ip-sec-tf-prototype/src/main/endpoint_cust_a.tf)
(plus a bonus `100.64.0.0/10 → VPC attachment` on the per-endpoint RT
for NF-edge return traffic).

When scaffolding `templates/endpoint-site/`, represent all 3 as separate
`pulumi.Route` / `TransitGatewayRoute` resources so onboarding = 3 adds
and offboarding = 3 removes. Static route #1 currently lives inline in
`hosting_region.tf`'s `aws_route_table.ziti_isolation` for convenience;
refactor to a separate `aws_route` (TF-side) / `aws.ec2.Route` (Pulumi-
side) for clean per-endpoint lifecycle.

### Shared-vs-per-endpoint breakdown (refined from NetEng)

The tier model in `ARCHITECTURE.md` is correct at the conceptual level,
but NetEng clarified the concrete sharing boundaries:

**Hosting Region / shared across endpoints (assuming no CIDR overlap):**
- VPC
- The IPsec subnet (one per AZ) — this is the subnet the TGW VPC
  attachment enrols
- The IPsec subnet's route table
- TGW itself
- TGW VPC attachment
- Shared TGW VPC route table (the "hub" RT, e.g. `ZT_HUB` in the
  prototype)
- VPC attachment → hub-RT association + propagation

**Per Endpoint Site:**
- Customer Gateway
- VPN Connection (TGW-attached)
- Per-endpoint TGW VPN route table (`Customer_A`, one per endpoint)
- VPN-attachment → per-endpoint-RT association + propagation (+ VPC
  attachment propagating into the per-endpoint RT too)
- The 3 static routes (each goes into a shared or per-endpoint RT but
  is *logically* owned by the endpoint)

If endpoints have overlapping CIDRs, you'd need a second TGW (AWS rule:
one subnet attachment per TGW per AZ) with its own shared-RT + VPC-
attachment, and the shared layer grows by one TGW's worth.

### Gotchas we hit on the TF import side

When porting to Pulumi, expect similar sharp edges:

1. **`aws_vpn_connection` (TF) / `aws.ec2.VpnConnection` (Pulumi)**: the
   auto-generator emits 0s for unset tunnel option integers (e.g.
   `tunnel1_rekey_margin_time_seconds = 0`), and the provider validator
   rejects them as out-of-range. Keep only the tunnel attributes that
   have real values (`inside_cidr`, `preshared_key` if you're pinning
   PSKs); let AWS defaults handle the rest.
2. **TGW VPN attachment is implicit.** You don't create
   `aws_ec2_transit_gateway_vpn_attachment` / equivalent as a separate
   resource — it's created as a side-effect of setting
   `transit_gateway_id` on the VPN Connection. In Pulumi, this is
   `pulumi.String(hub.id)` on `pulumi.ec2.VpnConnection{TransitGatewayId: ...}`.
3. **`aws_main_route_table_association`** doesn't support TF import
   blocks. Pulumi may or may not have the same limitation; if it does,
   skip it — the main RT is determined by AWS anyway, and TF/Pulumi
   don't need to own the binding unless you want to change which RT is
   main.
4. **Route-table routes**: use individual `aws_route` / `pulumi.ec2.Route`
   resources rather than inline `route { }` blocks on the RT. The inline
   form is authoritative (TF/Pulumi will remove un-declared routes),
   which makes per-endpoint add/remove messy. Separate route resources =
   per-route lifecycle, which is what you want.
5. **Default TGW association & propagation are disabled** on this TGW
   (`DefaultRouteTableAssociation: disable`,
   `DefaultRouteTablePropagation: disable`). Mirror that in your
   template or drift appears.
6. **`vpn_ecmp_support` default is `enable`**, but this TGW has it
   `disable`. Explicit `disable` in config to avoid drift.

See the prototype's `import-network-primitives` branch history for the
full set of edits that made `terraform plan -generate-config-out` round-
trip cleanly against the live AWS state.

## Target Pulumi structure

Mapping tiers → Pulumi templates + API endpoints:

| Tier | Template | API endpoint | Status |
|---|---|---|---|
| Platform | n/a (manually managed ECR + controller) | n/a | Exists outside Pulumi. |
| Hosting Region | **NEW** — `templates/hosting-region/` | `POST/GET/DELETE /hosting-regions` | **To be created.** VPC, ECS cluster, task-exec role, egress-any SG, state bucket + KMS. |
| Tenant | **NEW** — `templates/tenant/` | `POST/GET/DELETE /tenants` | **To be created.** Thin AWS footprint (tags, naming prefix, optional IAM boundary); primary work is to register the tenant's zrok account token as a Pulumi config secret consumed by Share stacks. |
| Endpoint Site | **NEW** — `templates/endpoint-site/` | `POST/GET/DELETE /endpoint-sites` | **To be created.** TGW, TGW VPC attachment, Customer Gateway, VPN Connection, TGW VPN attachment, TGW route table + routes, subnet in VPC, subnet route table, internal NLB (pinned IP), NLB SG, task ingress SG. |
| Share | [`templates/frontdoor-tcp-shares/`](./templates/frontdoor-tcp-shares/) | `/frontdoor-tcp-share` (exists) | **Refactor.** Already handles target group + listener + task def + service + log group. Extend to take Endpoint Site + Tenant outputs as stack references rather than current config plumbing, and drop the custom `frontdoor-mtls` vs `zrok` assumption now that the prototype locked in the image contract. |

Stack-reference dependency graph: Share → Endpoint Site → Tenant → Hosting Region → (manual Platform).

## Concrete porting map

For each resource in the prototype, here's where it should land:

| Prototype file | Resource | Target template |
|---|---|---|
| `ecs.tf` | `aws_ecs_cluster.this`, `aws_ecs_cluster_capacity_providers.this` | `templates/hosting-region/` |
| `iam.tf` | `aws_iam_role.task_execution`, `aws_iam_role_policy_attachment.task_execution_managed` | `templates/hosting-region/` |
| `security_groups.tf` | `aws_security_group.egress_all` + egress rule | `templates/hosting-region/` |
| `data.tf` | `aws_vpc` | Created by `templates/hosting-region/`, not data-sourced. |
| `data.tf` | `aws_ec2_transit_gateway`, `aws_subnet`, `aws_route_table` | Created by `templates/endpoint-site/`, not data-sourced. |
| `nlb.tf` | `aws_lb.this` (pinned IP) | `templates/endpoint-site/` |
| `security_groups.tf` | `aws_security_group.nlb`, `aws_security_group.tcp_share` | `templates/endpoint-site/` |
| `security_groups.tf` | `nlb_from_vpc`, `nlb_from_ipsec`, `tcp_share_from_vpc`, `tcp_share_from_ipsec`, `nlb_to_tcp_share` | Mostly Endpoint tier, except per-listener-port ingress rules which are Share-tier. |
| `logs.tf`, `ecs.tf` | `aws_cloudwatch_log_group.frontend`, `aws_ecs_task_definition.frontend`, `aws_ecs_service.frontend` | `templates/frontdoor-tcp-shares/` (already there; refactor to consume upstream stack refs). |
| `nlb.tf` | `aws_lb_target_group.frontend`, `aws_lb_listener.frontend` | `templates/frontdoor-tcp-shares/` (already there). |
| n/a | TGW / VPN / Customer Gateway / TGW attachments + RT | **NEW — Endpoint Site must add these**; NetEng currently hand-creates, and the handoff direction is that orchestration owns this. |

## Variables & env-var contracts

These are locked in by the prototype and the `frontdoor-mtls` image; don't
diverge without coordinating with the `frontdoor-mtls-cloud` team:

### Container env (Share tier task def)

| Env var | Scope | Source |
|---|---|---|
| `ENVIRONMENT` | Hosting Region (staging/production) | Stack config on Hosting Region stack |
| `FRONTDOOR_API_ENDPOINT` | Platform (e.g., `https://api-v2.frontdoor.staging.netfoundry.io`) | Hosting Region stack config (platform varies with env) |
| `FRONTDOOR_ENABLE_TOKEN` | Tenant | Tenant stack output (sensitive) |
| `FRONTDOOR_SHARE_TOKEN` | Share | Share stack config |
| `FRONTDOOR_BIND_ADDRESS` | Share (constant `0.0.0.0:9191`) | Share template |
| `FRONTDOOR_TLS_ENABLED` | Share (default `false` in prototype; `true` in production with cert material in Secrets Manager) | Share stack config |
| `FRONTDOOR_DEBUG` | Share (default `false`) | Share stack config |

If `FRONTDOOR_TLS_ENABLED=true`, the image needs `FRONTDOOR_TLS_CERT`,
`FRONTDOOR_TLS_KEY`, `FRONTDOOR_TLS_CLIENT_CA`,
`FRONTDOOR_TLS_PINNED_CLIENT_CERT`, `FRONTDOOR_TLS_REQUIRED_ORG_UNIT`,
`FRONTDOOR_TLS_ALLOWED_SUBJECTS`, `FRONTDOOR_TLS_ALLOWED_CN_PREFIXES`,
`FRONTDOOR_TLS_REQUIRE_CLIENT_CERT`, `FRONTDOOR_TLS_CRL_CHECK_ENABLED`,
`FRONTDOOR_TLS_CRL_FILE`. See the existing
`templates/frontdoor-tcp-shares/` secret plumbing for the pattern.

### Target group / NLB conventions (Share tier)

- Container port: `9191`.
- Protocol: TCP.
- `target_type = "ip"` (Fargate awsvpc).
- `preserve_client_ip = true`.
- `deregistration_delay = 30`.
- Health check: `interval = 30`, `timeout = 15`, `healthy_threshold = 2`,
  `unhealthy_threshold = 4`.

### ECS service conventions

- `launch_type = "FARGATE"`.
- `deployment_circuit_breaker { enable = true, rollback = true }`.
- `force_new_deployment = true`.
- `propagate_tags = "SERVICE"`.
- `enable_ecs_managed_tags = true`.
- `health_check_grace_period_seconds = 60` (needed because `zrok enable`
  runs at task boot and takes ~5-10s before the frontdoor process binds).
- Two SGs attached: the Endpoint Site's `tcp_share` SG + the Hosting
  Region's `egress_all` SG.

## Gotchas the prototype found (don't re-learn)

1. **`tls.enabled: false` requires `frontdoor-mtls` image that includes
   the optional-cert fix** (see `../frontdoor-mtls/CHANGE-support-disabled-tls.md`).
   Pinning to a pre-fix image tag causes `Config.TLS.CertFile: required
   field missing` on task startup.
2. **`FRONTDOOR_DEBUG=true` + `FRONTDOOR_TLS_ENABLED=false` is broken.**
   The entrypoint unconditionally `cat`s `certs/frontdoor-server.crt` in
   debug mode, which doesn't exist when TLS is off → task exits non-zero.
3. **Rootless podman port-forwarding rots.** Not relevant here (Pulumi →
   Fargate → host ENIs), but a warning for the Service Site side: backend
   echo servers or actual TCP services should bind directly on host ports
   (`--network=host`), not rely on rootless port proxies.
4. **`zrok2 share private -b tcpTunnel` target must be bare `host:port`**,
   not `tcp://host:port`. Happens at the Service Site, but if this project
   ever helps provision those, enforce the format.
5. **NLB security groups with `preserve_client_ip=true`:** task SG rules
   must be **CIDR-based**, not SG-referenced. The task sees the real
   customer IP, so a rule pointing at the NLB SG never matches.
6. **SG name collision on replacement:** if you ever want to force-replace
   a SG via description change, use `name_prefix` + `create_before_destroy`
   to avoid the deadlock the prototype hit (new SG can't be created
   because old still exists under the same name; old can't be deleted
   because other SG's rule still references it).
7. **Cross-account ECR pull** is already set up from `745435643501` to
   current workload accounts. Each new Hosting Region account will need
   the image-repo policy updated to add its account ID.

## Open questions (decide before landing)

Copied from `../ip-sec-tf-prototype/ARCHITECTURE.md`. Answers drive
template shape:

1. Is `FRONTDOOR_ENABLE_TOKEN` tenant-scoped (current assumption) or
   share-scoped? If share-scoped, move it from Tenant stack output to
   Share stack config.
2. Do we want an explicit "access grant" resource between Endpoint Site
   and Share to model "this Endpoint is allowed to reach this Share,"
   rather than expressing it implicitly as an SG rule? If yes, add a
   fifth template (e.g., `templates/share-grant/`) or a sub-resource
   inside the Endpoint Site stack.
3. VPC: shared across Tenants (current recommendation) or per-Tenant?
   Confirm against any compliance constraints before committing.
4. Egress path from Fargate: IGW + public-IP-per-task (prototype) or NAT
   Gateway + private-IP? Hosting Region tier decision.
5. Who owns the S3 state bucket creation — is it still `pulumi/` (shared
   infra) or a bootstrap output of the first Hosting Region? Today
   `nf-pulumi-state` is effectively global.

## Suggested sequence

1. **Read** `../ip-sec-tf-prototype/ARCHITECTURE.md` end-to-end. This
   handoff is a skimmer; the architecture doc is the real spec.
2. **Skim** the prototype's `.tf` files (`nlb.tf`, `ecs.tf`,
   `security_groups.tf`, `logs.tf`, `iam.tf`, `data.tf`, `variables.tf`,
   `outputs.tf`) as the working implementation reference.
3. **Answer the open questions** (at least 1 and 2 — they affect
   template count and wiring).
4. **Scaffold** `templates/hosting-region/` first. Pattern after
   `templates/frontdoor-tcp-shares/` (go.mod, main.go, provider.go,
   outputs.go), minus the LB-specific pieces. Provide outputs the next
   tier consumes: vpcId, vpcCidr, clusterName, taskExecRoleArn,
   egressSgId, stateBucketName.
5. **Scaffold** `templates/endpoint-site/`. Consumes Hosting Region
   outputs via stack reference. Outputs: nlbArn, nlbDnsName,
   nlbPrivateIp, tcpShareSgId, subnetId, endpointCidr (for SG rules in
   Share). Manage TGW / Customer Gateway / VPN Connection etc. here.
6. **Scaffold** `templates/tenant/` (small). Mostly tags and the
   `FRONTDOOR_ENABLE_TOKEN` stored as a Pulumi secret output.
7. **Refactor** `templates/frontdoor-tcp-shares/` to consume Endpoint
   Site + Tenant outputs instead of taking all config directly. Most of
   its current logic stays; the inputs change.
8. **Add controllers + API endpoints** in `controllers/` for the three
   new stack types. Follow the existing `/stacks` and `/frontdoor-tcp-share`
   pattern.
9. **Validate** the templates by deploying a fresh Hosting Region +
   Endpoint + Share into a test account/region. The prototype's
   environment is throwaway — don't try to `pulumi import` its
   resources; deploy clean from the templates and retire the prototype
   once the green-field deployment passes the same connectivity tests.

## Cross-repo coordination checklist

- Before making changes here that affect the container contract, check
  `../frontdoor-mtls-cloud/frontdoor_mtls_entrypoint.py` and
  `../frontdoor-mtls/config.go` for the expected env-var and YAML
  shape. Any mismatch surfaces as a crash-looping task.
- Before changing the tier model, update all three artifacts in the
  living triumvirate (see `../ip-sec-tf-prototype/ARCHITECTURE.md`
  front-matter): `ARCHITECTURE.md`, `diagrams/tiers.dot` + renders,
  and the Confluence page.
- When bumping `frontdoor-mtls` image tag used by the Share tier, test
  against the prototype first (it's cheap, fast, single-customer, and
  has a working zrok loopback validator on the backend host at
  `100.55.0.139`). If it's green there, promote the tag here.

## Non-goals for this handoff

- Re-deriving the tier model. Don't argue it; argue the open questions
  above if you disagree with a choice.
- Decommissioning the `ip-sec-tf-prototype` repo. That stays as a
  smoke-test playground during the port; retired only once
  zrok-orchestration can stand up a clean Endpoint + Share and prove
  the same end-to-end `nc` flow against the pinned NLB IP.
- Writing the `zrok share` / backend side. Those live at the Service
  Site (tenant's infra). This repo only manages the **ingress** half.