Post-quantum TLS key exchange on Cilium Gateway API

We host autonomous companies, AI agents that run a business rather than assist with one, on hardware we operate ourselves in the EU. We terminate our own TLS, which makes the choice of key exchange ours to make and ours to answer for. The EU’s coordinated roadmap for post-quantum cryptography puts public-facing TLS in the first migration wave, with national plans due by the end of 2026. And unlike most security work, this part cannot be done retroactively: traffic captured today is already sitting on somebody’s disk, and it stays readable the moment the key exchange protecting it stops holding.

So we went to turn on hybrid post-quantum key exchange at the edge, and found there was no way to ask for it.

Cilium’s Gateway API generates the Envoy configuration for our ingress, and it has no field for TLS curves. No tls_params, no ecdh_curves, no supported way to add either to a Gateway-managed listener. The upstream request for one, cilium#38089, used post-quantum key exchange as its own motivating example. It got no maintainer reply and was auto-closed as stale.

We got X25519MLKEM768 onto the edge anyway, with a mutating admission webhook. It works for a reason that took us a while to believe: the Cilium operator’s habit of reverting your edits within a second is the same mechanism that ends up applying them.

Why key exchange, and not the rest of TLS

What sets the priority is whether an attack has to happen live or can be run later against a recording.

A forged signature has to be used in the moment, against someone who is watching, while the certificate is still valid. A broken key exchange is retroactive: the attacker only needs to have been recording. That traffic is already on a disk somewhere, and it stays there, waiting for the arithmetic to get cheaper. Key exchange is the part with a deadline attached, which is why post-quantum TLS rollouts start there and why ours did.

Symmetric encryption is not in scope. Grover’s algorithm gives a quadratic speedup that does not parallelise usefully, which is why NIST still benchmarks Category 1 security against AES-128. AES-256, ChaCha20-Poly1305 and HMAC-SHA256 are unaffected.

Hybrid here means a TLS 1.3 handshake deriving its shared secret from both X25519 and ML-KEM-768, standardised by NIST as FIPS 203, so the session survives unless both are broken. Worth knowing if you are reading version strings: X25519MLKEM768 is not X25519Kyber768 and the two do not interoperate. The latter was the earlier draft hybrid built on pre-standard Kyber.

Hybrid rather than pure post-quantum is a deliberate choice, and for us a required one. Germany’s BSI asks for it explicitly in TR-02102-1: post-quantum combined with a classical algorithm, not post-quantum alone. Under that guidance hybrid is the destination rather than a waypoint on the road to something purer, which also makes the fallback behaviour below a feature rather than a compromise.

Envoy could do it. Cilium had no way to ask.

We terminate TLS at Envoy, managed by Cilium’s Gateway API implementation, on the Talos and Hetzner cluster we autoscale with our own Karpenter provider. Envoy’s TLS stack is BoringSSL, which has supported the hybrid group for a while. Nothing was missing at the bottom of the stack.

The gap is in the translation layer. Cilium derives a CiliumEnvoyConfig from the Gateway resource, and that derivation covers ALPN, proxy protocol and socket options, but not tls_params, so ecdh_curves has nowhere to come from.

The frustrating part is that Gateway API already specifies where this should live. A listener’s tls.options is a map of implementation-specific key/value pairs, and the field’s own doc comment gives the example: “configuring the minimum TLS version or supported cipher suites”, with implementation-specific keys required to be domain-prefixed. Cilium does not read it. Its ingestion in operator/pkg/model/ingestion/gateway.go passes the listener’s TLS block to a toTLS helper that resolves certificate references and nothing else, and the strings Options, cipherSuite and ecdhCurves do not appear in that file at all. So the escape hatch exists in the API and is unimplemented in this controller, which is a different problem from the one we assumed we had.

The documented workaround is to take the Gateway away from the operator and hand-maintain the generated config, which means also hand-maintaining the load balancer and certificate wiring that come with it. We did not want to own that forever for the sake of one field.

Our first survey of the problem, in June, went from “the Kubernetes 1.34 MutatingAdmissionPolicy we want is still alpha” straight to building a custom Cilium operator image or standing up a separate OpenSSL proxy at the edge. Both were bad. Both were also unnecessary: a plain MutatingWebhookConfiguration has been GA for years and injects at the same point in the request path. The mechanism was never exotic. We just walked past it.

The operator reverts your patch, and that turns out to be the seam

The obvious move is to edit the generated CiliumEnvoyConfig in place. That fails, but interestingly. The patch is accepted, and the field is already gone when you read it back 0.6 seconds later.

Our own engineering notes from that attempt recorded two things that cannot both be true: that the Cilium operator strips the field, and that the resource retains it. Somebody had written down the symptom twice from different angles and moved on. Resolving that contradiction mattered more than it looked, because three different explanations fit the evidence, and they pointed in opposite directions.

Rows two and three would have sunk the approach. They would also have sunk the fallback we were holding in reserve, the 1.34 MutatingAdmissionPolicy, because it injects at exactly the same point. Taking either on trust would have bought us a one-way cluster upgrade and a full rolling node reboot to reach a mechanism that could not have worked.

Both experiments were cheap to run. The answer is row one: the operator reverts drift within about a second, and nothing else touches the field.

Which reframes the problem. A controller that reliably rewrites a resource is a controller whose writes pass through admission control, and that holds for any Envoy setting Cilium does not expose, not just this one. The test worth running first is whether the controller genuinely rewrites the resource or merely validates it, because only the first gives you a write to intercept.

Injecting ecdh_curves at admission time

The webhook watches CiliumEnvoyConfig writes in the gateway namespace and patches the one the Gateway generates, injecting the curve preference into every filter chain carrying a TLS context. The operator regenerates the config, the regeneration passes through admission on the way in, and it lands already carrying the setting.

It is 163 lines of Go with no third-party dependencies, and it is on GitHub at paperclipinc/pqc-admission-webhook under Apache-2.0, with the manifests and the forced-write step written up. If you are on Kubernetes 1.34 or newer you do not want it, because the same injection is expressible declaratively as a MutatingAdmissionPolicy with no service to run.

What ends up in the TLS context:

commonTlsContext:
  tlsParams:
    ecdhCurves:
      - X25519MLKEM768   # hybrid post-quantum
      - X25519           # classical fallback
      - P-256

And the registration:

webhooks:
  - name: pqc.paperclip.inc
    failurePolicy: Ignore
    sideEffects: None
    reinvocationPolicy: IfNeeded
    rules:
      - apiGroups: ["cilium.io"]
        apiVersions: ["v2"]
        resources: ["ciliumenvoyconfigs"]
        operations: ["CREATE", "UPDATE"]

failurePolicy: Ignore is doing real work there, along with the fact that every path through the handler returns an allowing response, including on input it cannot parse. A webhook sitting in front of the resource that configures your public ingress is a good way to take your own site down. This one cannot: if it is broken, writes pass through unmodified, the stored setting stays where it is, and we lose post-quantum key exchange only at the next regeneration. It can degrade our crypto. It cannot stop our front door from being configured.

Which is the right trade and also the dangerous one, because it means the failure is silent. A dead webhook produces no error and no outage. It produces a page that still loads, over a handshake that has quietly gone back to classical X25519, while this post and our security documentation carry on claiming otherwise. We shipped it that way and left it that way for longer than we should have: for the first weeks the only check was a human running the verification command below, which means the person most likely to notice a regression was a reader. There is now a daily canary that offers the hybrid group and nothing else and fails loudly if the handshake does not complete.

The curve list keeps X25519 and P-256 behind the hybrid group. A hybrid-only list would hard-fail every client without ML-KEM support, which is the traditional way to convert a security upgrade into an outage.

The one implementation detail we would flag to anyone copying this: find the TLS chain by inspection rather than by index. Indexing the filter chain by position works right up until a listener is added or reordered, and then it quietly stops matching and drops the edge back to classical with nothing to indicate it. A control that fails silently into a weaker state is worse than one that fails loudly, because you keep the belief without the property. The webhook patches every chain carrying a TLS context, and skips any that already has the setting so repeated calls cannot clobber it.

Two things we checked before pointing it at production. The obvious risk was a write loop: the operator wants the field absent, the webhook keeps putting it back, and the two grind against etcd forever. Measured over 60 seconds, the resource version does not move. The second was simply whether it worked at all, so we pointed it at a throwaway config object that routed nothing, confirmed injection there, and confirmed the real one was untouched.

Then the cutover did nothing at all.

This is the part worth the price of admission if you are reproducing any of it. Nudging the Gateway to force a reconcile does not help, because if the config the operator derives is byte-identical to the stored one it issues no write at all. No write, no admission request, nothing to mutate. You need a real write. We got one by patching the stored config so the operator would revert it, and that revert was itself the write that came back through the webhook carrying the curves.

Verified on Cilium 1.18.10 with cilium-envoy v1.36.6, on Kubernetes 1.33. Envoy’s own documentation does not list X25519MLKEM768 among supported curves, so check your build rather than trusting the docs.

What it costs

ML-KEM-768 keys are much larger than elliptic curve keys, so the handshake grows. These numbers are measured against our own edge, which is worth saying because an earlier version of this post measured cloudflare.com instead: at the time, our edge did not yet support the group, so there was nothing of ours to measure.

200 handshakes per arm, arms interleaved so any drift over the run hits both equally, timing the TLS handshake on an already established TCP connection. Process startup and a roughly 23 ms TCP round trip are excluded deliberately, because leaving either in buries an effect this size.

The byte cost is exact, and it is barely a measurement. The extra 1184 bytes outbound is the ML-KEM-768 encapsulation key and the extra 1088 inbound is the ciphertext, both fixed by FIPS 203. About 2.2 KB more per handshake, every handshake, and no amount of tuning at either end will improve on it.

The time cost is about a millisecond, close to 3% of a 31 ms handshake. Four separate runs put it at +0.15, +1.12, +0.93 and +0.97 ms.

What we cannot tell you is what happens in the tail, and an earlier version of this post claimed we could. It said the tail widens by roughly 20% at p90 and p99. We no longer stand behind that. Running the same experiment four times against our own edge, the p99 difference came out -10.7, +11.6, -26.6 and +3.4 ms. It does not hold its sign, let alone its magnitude. At 200 samples from one client, a p99 is measuring the network between us and not the key exchange, and the earlier figure was a single run of exactly that read too confidently. Cloudflare’s measured few-percent slowdown at internet scale is the number to trust, and citing it alone would have been the better call.

The byte count has a second-order effect that matters more than the milliseconds. A 1.2 KB key_share pushes the ClientHello past a single packet, so it splits across segments. Mostly uninteresting over TCP. Genuinely interesting for QUIC, and for middleboxes that mishandle a fragmented hello.

The byte count has a second-order effect that matters more than the milliseconds. A 1.2 KB key_share pushes the ClientHello past a single packet, so it splits across segments. Mostly uninteresting over TCP. Genuinely interesting for QUIC, and for middleboxes that mishandle a fragmented hello.

What this does not protect

Our certificates are not post-quantum. No publicly trusted CA issues ML-DSA certificates as of July 2026, and ours is ECDSA P-256 from Let’s Encrypt, which is entirely classical. So the key exchange is protected and the signature proving we are who we say we are is not. That ordering is deliberate, for the live-versus-recorded reason above, but “post-quantum key exchange” and “post-quantum TLS” are different claims and we are only making the first, here and in our security documentation.

Node-to-node key exchange is also still classical. Traffic between nodes in our network-isolated cluster is encrypted with WireGuard, whose ChaCha20-Poly1305 cipher is quantum-resistant, but whose key exchange is X25519. WireGuard’s pre-shared key option would close that gap, except Cilium manages its WireGuard keys internally and exposes no hook to set one, so this is somebody else’s roadmap rather than ours.

On traffic to model providers we claim nothing at all. Agents reach provider APIs directly using your own key, over whatever those providers negotiate, and that far end is not ours.

The rollout also could not be rehearsed on staging, which is worth admitting because it is not how we normally work. We have one Gateway and one generated config, shared between staging and production, so the usual rule of proving a change on staging first did not apply. We compensated structurally instead: fail open, scope to a single named object, rehearse against a config that routes nothing, keep rollback to deleting one object.

Checking it yourself

Do not take the headline claim on trust. With OpenSSL 3.5 or later:

# Offer ONLY the hybrid group. A successful handshake proves it is negotiated.
openssl s_client -connect paperclip.inc:443 -servername paperclip.inc \
  -groups X25519MLKEM768 -brief </dev/null

# Confirm classical clients still work.
openssl s_client -connect paperclip.inc:443 -servername paperclip.inc \
  -groups X25519:P-256 -brief </dev/null

The first should establish a TLS 1.3 connection rather than fail. Two traps on the client side, both of which produce failures that tell you about your machine rather than about our server: anything before OpenSSL 3.5 has no ML-KEM support at all, and the openssl on macOS is LibreSSL, which rejects the group name outright. Keep the </dev/null as well, or s_client connects and then sits there waiting on your keyboard.

The equivalent check inside a cluster is whether the setting survived a regeneration:

kubectl -n gateway get ciliumenvoyconfig <name> -o yaml | grep -A4 ecdhCurves

Chrome has negotiated X25519MLKEM768 by default since 131 and Firefox since 132, so if your browser is current, the connection that delivered this page was almost certainly protected by post-quantum key exchange already.