#kernel optimization

RunAnywhere open-sourced an AWS Trainium2 inference engine for GLM-5.3-Flash, a 320B Mixture-of-Experts model with a 1M-token context, that serves the model on one trn2.48xlarge and rewrites each layer half as a single NKI kernel, or each whole layer for sparse-attention layers. On the same layout, a batch-8 decode step fell from 32.2 ms in the 17 October build to 24.2 ms on 1 November; the author attributes the gap to long chains of small vector-engine instructions at about 153 ns fixed cost each, HBM writes at every kernel boundary, and 125 cross-chip collectives per step. The remaining 24.2 ms includes a 6.1 ms memory floor, 10.0 ms of engines busy beyond it, 4.6 ms of waits inside kernels, and 2.8 ms of exposed collectives. An approximation remains in sparse-attention prefill on 1.2% of selections, shifting next-token log-probabilities by 0.012 to 0.021 nats on average and changing the top token at 3 of 448 positions; no task-level evaluation has been run on this build.