Open Item Disposition: S1 (ACA silo↔silo) and BENCH-04 (per-room geometry cost)
Open Item Disposition: S1 (ACA silo↔silo) and BENCH-04 (per-room geometry cost)
Date: 2026-08-01 · Status: S1’s fallback has been ruled in advance (the project is no longer blocked by it); BENCH-04’s analytical upper bound has been recomputed, correcting an error in W1 Depends on:
Architecture_Decision_Rulings_R1.md(ADR-032),W1_Tick_Budget_Revisions.md(Tables A/B, Q-015, Q-016) Produces: ADR-062 (the S1 fallback topology, Conditional), the correction to Q-015/Q-016, and executable specifications with pre-registered criteria for both spikes
0. Why These Two Need Different Handling
Both are empirical questions, but they are opposite in character:
- S1 is binary, external, testable now, and enormously consequential if negative (Orleans multi-silo unusable → the entire elasticity argument of §10.1 rewritten). The correct handling for it is not to wait for the answer, but to design in advance a path that holds under both answers.
- BENCH-04 is continuous, internal, and not yet measurable (the Rust crate does not exist yet), but its analytical upper bound can be computed now, and the result overturns one of W1’s numbers. The correct handling for it is to fix the analytical estimate first, then demote the benchmark to “verifying an analytical estimate” rather than “discovering an unknown”.
Part One: S1 — Ruling the Fallback in Advance
The specific question to be verified
Orleans silo↔silo requires each silo to advertise a reachable IP:port and connect directly to a specific silo (default silo port 11111, gateway port 30000). Three sub-questions:
- Do ACA replicas obtain subnet IPs under VNet integration, and can replica A open a TCP connection directly to replica B’s IP?
- Does ACA’s internal ingress provide only load balancing to any replica (if so, it cannot be used for silo addressing)?
- Are arbitrary non-HTTP TCP ports reachable inside the environment (ACA’s ingress model is HTTP-centric)?
Experiment design (half a day, requiring no project code)
A minimal container image, two replicas:
- Each replica writes its own pod IP to shared storage on startup (one Neon table suffices).
- Each replica reads the other’s IP, opens a bare TCP connection to the other’s port 11111, and exchanges one byte.
- Record: connection success/failure, failure type (refused/timeout/DNS), handshake latency, and whether the IP changes within 30 minutes due to replica relocation.
- Control group: the same test between two separate ACA apps at
minReplicas=maxReplicas=1, over their respective internal FQDNs.
That control group is the key — it simultaneously verifies whether the fallback is viable, so whatever the main test returns, this spike produces a workable topology.
Executable implementation (added 2026-08-01)
The design above is conceptual — “a minimal container image”, “one Neon table suffices”. Nobody has run it, and what was missing is exactly the things it literally asks for and that do not exist. These are now supplied in spikes/S1_aca_silo_reachability/:
| File | Contents |
|---|---|
probe.py | The measurement tool. Listener + registrar + prober in one; a single image serves both the main test (DISCOVERY=rendezvous, against pod IPs) and the control group (DISCOVERY=static, against internal FQDNs) |
Dockerfile | Three lines, python:3.12-slim |
README.md | Step-by-step az commands, cost and teardown, and what has been verified and what has not |
Deliberately az CLI rather than Bicep/Terraform: this spike is disposable, and teardown is one az group delete. ADR-062 clause 5’s “scaling is IaC-driven” concerns the production topology, not a half-day probe.
The criteria table is not in that directory, only a link back to this section. A second copy of the criteria table would drift from this section, and the drifted copy would be exactly the one a decision was based on. Results are also written back into this section rather than opening a separate results document.
The probe has been verified locally: two instances discovered each other via rendezvous, completed the TCP handshake, and reported round-trip latency, with the inbound event confirming the listening side independently of the probing side’s own success claim. The six failure classification mappings are verified by unit tests. Not verified: every az command (no Azure CLI or subscription locally), the Docker build (no daemon), and the end-to-end refused path (a closed port on Windows reports timeout rather than refused; ACA runs Linux and that path should work, but that is inference rather than observation).
Criteria (pre-registered, to prevent post-hoc interpretation)
| Outcome | Verdict | Action |
|---|---|---|
| Main test succeeds and is stable for 30 minutes | S1-PASS | ADR-032 is finalized as written |
| Main test succeeds but the IP changes on replica relocation (expected) | S1-PASS-CONDITIONAL | ADR-032 is finalized, but the silo advertised address MUST be re-registered on every startup, and §10.1.1’s “silo↔silo is a release verification item” is promoted to a continuous health check (automatically verified after every deployment, not once) |
| Main test fails (any failure type) | S1-FAIL | Activate ADR-062’s silo-per-app topology |
| The control group also fails | S1-FAIL-HARD | ACA is unsuitable for hosting Orleans. Only then does §10.1’s platform choice need re-examining (rather than re-examining it now) |
ADR-062 — Silo-Per-App Topology (activated on S1-FAIL)
Status: Conditional (in force only on S1-FAIL; otherwise permanently Proposed) · Date: 2026-08-01 · Amends: ADR-032, ADR-002 (clarifies)
Context
If ACA replicas cannot address each other over TCP directly, the topology of “one app, N replicas, one silo per replica” cannot form an Orleans cluster. But ACA’s internal ingress provides a stable FQDN per app, which is exactly what silo addressing needs — merely at a different granularity.
Decision
- One silo = one ACA app, with
minReplicasandmaxReplicasboth 1, all deployed from the same OCI image (therefore still fully consistent with ADR-002’s Modular Monolith: horizontal scaling of one deployable unit, not a microservice split). - Each silo therefore has a stable, addressable internal FQDN, and Orleans advertises that FQDN rather than a pod IP. silo↔silo goes over the internal ingress and needs no pod-level reachability.
- An explicit room→silo assignment table, authoritatively stored in the clustering branch (which per ADR-049 never suspends):
- Orleans’ placement director (the weighted director of §4.4) writes that table rather than deciding only in memory;
- the edge routing layer (a Cloudflare Worker + KV cache) reads that table, routing a
RoomIdto the corresponding app’s FQDN. - Placement and routing are therefore driven by one source of truth, structurally eliminating the “two components each owning placement” that F-01 identified — the key improvement of this fallback over ADR-024: ADR-024 assumed the edge could infer placement, while this design has the edge read placement.
- ADR-032’s targeted forwarding is retained as a safety net, not replaced: it covers the window in which “the room has been reassigned but the edge cache has not refreshed”. The two mechanisms combine rather than compete.
- Scaling is IaC-driven rather than autoscaler-driven: adding capacity = deploying a new app (a Terraform/Bicep count parameter). Each app can still scale to zero independently, so what is obtained is scale-to-zero at room-group granularity rather than for the whole tier.
Alternatives Considered and Why Rejected
- Deterministic
hash(RoomId)→siloplacement, with the edge routing by the same function: needs no assignment table and is the simplest; but a hash cannot load-balance, directly killing the reason §4.4 exists (count-based placement packs heavy rooms into the same silo after a scale-out). The assignment table preserves weighted placement. - Switch to AKS: ADR-002 explicitly rejects K8s, and this fallback holds within ACA, so there is no reason to overturn a core decision for it.
- Abandon multi-silo and use a single larger machine: room capacity is bounded by a single host’s tick budget, which is infeasible at SaaS scale. (Amended by ADR-108, 2026-08-06: this cited §10.1’s Self-Hosted profile as the worked illustration of that bound. The profile was removed; the bound was never a concession to it — it is a property of one Orleans activation, §5.2 — so the rejection stands with its illustration retired.)
- Pull the backplane back onto the correctness path: explicitly forbidden by §10.1.1(2) rule (1).
Consequences (including negative)
- Scaling granularity becomes coarser: adding capacity is a deployment action rather than autoscaling. For the known and predictable pattern of “weekend evening peaks”, scheduled IaC application suffices; for traffic spikes the response is slower. This is a real loss of elasticity.
- The edge routing layer is a new component: a Cloudflare Worker + KV. It is small, cheap, and Cloudflare is already in the stack (§10.1), but it is a new failure path and needs a fail-safe (on a KV miss or expiry, fall back to “any silo + ADR-032 targeted forwarding” rather than refusing the connection).
- The app count becomes an IaC-managed capacity parameter, requiring an explicit scaling runbook and room reassignment tooling.
- Each app has its own cold start; scale-to-zero at room-group granularity means waking one group wakes that group’s entire silo (coarser than per-room).
This looks like a violation of ADR-032, and it is not — the difference is who owns the table (R3-F-06).
The core sentence by which ADR-032 rejects ADR-024 is: “any mapping held by the edge layer is stale by construction”. And this ADR’s mechanism is precisely a room→silo assignment table read by the edge router. On the surface it is the same thing.
The actual difference is the writer:
| Who decides placement | Who holds the mapping | Result | |
|---|---|---|---|
| ADR-024 (rejected) | §4.4’s weighted director, independently | the edge layer infers for itself | the inferrer is not the decider, so the mapping necessarily lags the fact it tries to describe |
| ADR-062 (this one) | the placement director | the edge layer reads the table the director publishes | the writer is the decider, so the mapping is authoritative rather than speculative |
ADR-032’s criticism targets the former. It does not apply to the latter, and that MUST be written here — otherwise a future reader comparing ADR-032’s sentence with this mechanism would reasonably conclude this ADR violates it, and that conclusion would be used, under the pressure of an S1-FAIL, to argue against the only viable fallback.
Incidentally, this is also why the fail-safe above (on a KV miss, fall back to “any silo + ADR-032 targeted forwarding”) is necessary rather than merely conservative: it makes the edge table an optimization rather than a correctness dependency, exactly isomorphic to ADR-032’s treatment of sticky ingress.
Enforcement
- Post-deployment automatic verification: internal FQDN reachability between any two silo apps (a continuous health check, not a one-off).
- Integration test: after a room reassignment, delivery remains correct while the edge cache is unrefreshed (the ADR-032 safety net working).
- Metrics:
EdgeRouteCacheMiss,RoomReassignmentLatency, and the room weight distribution per app.
§11 Index Line
ADR-062 (Conditional, active only on S1-FAIL): One silo per ACA app (single replica, same image) so each silo has a stable internal FQDN; an explicit room→silo assignment table in the always-on clustering branch is written by the placement director and read by the edge router, so placement and routing share one source of truth. ADR-032's targeted forwarding remains the safety net for reassignment windows. (Amends ADR-032; clarifies ADR-002.)
Part Two: BENCH-04 — Recomputing the Analytical Upper Bound and Correcting W1
Correction: Q-015/Q-016 mistook a “budget ceiling” for a “capacity unit”
W1 Table B gives ⊙0.85 cores per room and defines Q-016 from it (1 room weight = a full-budget room), and on that basis I told you “500 rooms at full budget need about 425 cores”. That inference is wrong, and the error is summing each item’s budget ceiling as though it were the expected cost. The purpose of a budget ceiling is to isolate pathological rooms, not to describe a typical one.
Recomputing the expected cost:
Geometry (originally estimated ⊙300ms/s, i.e. 0.3 cores)
- A ⊙30m sight radius with 1m cells → roughly 2,800 cells of visibility computation per observer.
- ADR-048’s grouping policy makes the unit of computation the group rather than the player: the reference scenario of 50 players in 4 teams gives ⊙5 groups.
- Only the groups that moved need recomputing each tick (at TRPG pace, most ticks have nobody moving). Pessimistically assuming 2 groups recompute per tick: 2 × 2,800 cells × ⊙20ns (including the occluder spatial hash lookup, mostly cache hits) ≈ ⊙0.11ms/tick.
- BVH refit (512 dynamic occluders): 512 × log₂512 ≈ 4,600 ops ≈ negligible.
- A*: not per tick; on demand.
- Expected ≈ ⊙0.15ms/tick = 3ms/s = 0.003 cores per room. A factor of 100 below the 300ms/s budget ceiling.
The Grain thread (W1 Table A budgets 28ms/50ms)
- Segment B expects ⊙3ms (already derived item by item in W1, for the 50-viewer scenario); an 8-player room is roughly ⊙0.6ms.
- Segment A (Effect application): near 0 on ticks with no macro triggered; ⊙3ms only in the 1,000-Effect ceiling scenario.
- Intents: one every few seconds in an 8-player room, so 0 on most ticks.
- An 8-player room expects ≈ ⊙0.8ms/50ms = 0.016 cores; a 50-player room expects ≈ ⊙3.5ms/50ms = 0.07 cores.
Corrected expected cost per room
| Room type | Expected | Original W1 value |
|---|---|---|
| 8-player room | ⊙0.02 cores | 0.85 (full budget) |
| 50-player room | ⊙0.08 cores | 0.85 (full budget) |
500 8-player rooms ≈ ⊙10 cores, not 425 cores. The number I gave you earlier overestimates by roughly 40×, and would lead to serious over-provisioning in §10.2’s cost model and the silo specification.
This correction brings out two architectural observations
- The dominant capacity cost is viewer count, not world complexity. Segment B (per-viewer snapshot assembly) accounts for ⊙95% of the expected cost in a 50-player room, and geometry for ⊙4%. The current §5.2 table gives 15ms to geometry and 10ms to snapshots, weighting them opposite to the actual cost drivers. The corrected budget allocation should reflect this: snapshot assembly deserves more budget and more optimization attention (W1/D-2’s contiguous version array requirement is the most important item on that path), and geometry’s budget can be cut substantially and used as an isolation ceiling for pathological worlds.
- ADR-048’s grouping policy is why geometry is cheap. Keeping one visibility set per player (50 sets) would raise geometry cost roughly 10× to ⊙0.03 cores per room — still not high, but F-09’s memory problem would hit the wall first. This reinforces ADR-048: it solves memory and computation at once.
Corrected definitions of Q-015/Q-016
| ID | Old | New |
|---|---|---|
| Q-015 | GEOMETRY_POOL_MS_PER_ROOM_SEC = 300 (treated as a capacity estimate) | Split into two: GEOMETRY_CEILING_MS_PER_ROOM_SEC = 300 (an isolation ceiling; reaching it means the room is pathological — record and degrade); GEOMETRY_EXPECTED_MS_PER_ROOM_SEC = ⊙3 (for capacity planning) |
| Q-016 | ROOM_WEIGHT_UNIT = a full-budget room ≈ 0.85 cores | Redefined: ROOM_WEIGHT is the expected cost computed from viewer count (8 players ⊙0.02, 50 players ⊙0.08 cores), with a safety factor (⊙3× recommended) as the silo admission threshold. A budget ceiling is not a weight unit. |
§4.4’s admission ceiling is therefore expressed as “expected cost × safety factor”, while the per-room budget ceilings (Tables A/B) continue to serve as the isolation mechanism. The two are different things and were previously conflated.
BENCH-04’s role therefore changes
It is demoted from “discovering an unknown” to “verifying an analytical estimate”, and gains pre-registered criteria:
| Measurement | Analytical prediction | Criterion |
|---|---|---|
| Visibility recomputation for a single group over 2,800 cells (reference scene: 512 dynamic occluders, 16 resident chunks) | ⊙0.055ms | within 0.5×–3× → confirmed; > 3× → recompute the capacity model; > 10× → re-examine whether ADR-048’s grouping is sufficient |
| 50-viewer Segment B (covered by BENCH-03) | ⊙3ms | as above. > 3× means W1 Table A’s 5ms budget does not hold and needs reallocating |
| Expected cores per room (8 players / 50 players) | ⊙0.02 / ⊙0.08 | decides Q-016 and §10.2’s cost model |
| Contiguous version array vs dictionary baseline (BENCH-02) | a 5×–25× gap | if the gap is < 3×, W1/D-2’s normative requirement can be demoted to a recommendation |
BENCH-04’s priority therefore drops (what it verifies is an estimate that no longer blocks any ruling), and BENCH-02/BENCH-03’s priority rises — because the dominant cost is in Segment B, and whether Segment B’s 5ms budget holds depends on the baseline’s storage layout.
Final Disposition of Both
| Item | Disposition | Still blocking a ruling? |
|---|---|---|
| S1 | The experiment specification and control group are settled; each of the four outcomes has a pre-registered action; the S1-FAIL fallback ADR-062 is fully ruled | No longer blocking. The only outcome still causing a major rewrite is S1-FAIL-HARD (the control group also failing), and the control group uses ACA’s most basic internal ingress capability, so the probability of failure is low |
| BENCH-04 | The analytical upper bound is computed; Q-015/Q-016 are corrected; the benchmark is demoted to a verification | No longer blocking. But BENCH-02/BENCH-03 are promoted to first priority — they decide whether W1 Table A’s Segment B budget holds, and that budget is the actual capacity bottleneck |
Documents Requiring a Write-Back
W1_Tick_Budget_Revisions.md: the Q-015 split in Table B, the redefinition of Q-016, and an explicit distinction between “budget ceiling ≠ capacity unit”; Tables A/B each gain an “expected” column alongside “ceiling”.W6_Governance_And_Closure.md: update Q-015/Q-016 in the quantity registry; in the remaining open items, demote BENCH-04 and promote BENCH-02/03; change S1’s “if negative, rewrite §10.1” to “if negative, activate ADR-062”.Architecture_Decision_Rulings_R1.md: add ADR-062 (Conditional); update S1’s consequences column.- When the whitepaper is revised: §5.2’s budget allocation weights MUST reflect “viewer-dominated, geometry secondary” rather than the current opposite weighting. This is the most substantial effect of this recomputation on the whitepaper’s content.