"Compression is prediction" was the highest-scoring technical explainer of the cycle at 596 points.

Annie Sexton of ngrok grounds the compression/language-modeling equivalence in arithmetic coding: a no-context model needs 2.59 bits/symbol, an order-1 model 1.16 on the same text, and GPT-2 compresses Dickens to about 10% of original size against 24% for the order-1 baseline (ngrok). The argument is that LLMs minimize exactly the cross-entropy that sets the compression limit, making them the same mathematical object. 243 comments.