Geometric Counting in Claude 3.5 Haiku: How LLMs Learn to Line Break

Intellectually Curious

We pull back the curtain on how Haiku doesn’t simply “count” characters but builds a multi‑dimensional geometry: a curved count map (feature manifold) in high‑dimensional space, boundary heads that twist to align with the line width, and orthogonal representations that turn the fit decision into a simple linear separation. We also examine a surprising vulnerability—how a couple of at signs in prompts can hijack the counting rule—showing how perceptual these internal mechanisms are. Sponsored by embersilk.com.

Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information.

Sponsored by Embersilk LLC