HamJEPA: Hamiltonian Geometry Improves JEPA Representations by 4-10 Points on Standard Benchmarks

arXiv·low signal

HamJEPA replaces the standard isotropic Gaussian regularization in JEPAs with Hamiltonian dynamics and symplectic prediction, yielding +4.89 to +10.64 point improvements on kNN and linear-probe evaluations across CIFAR-100 and ImageNet-100. The key insight: when downstream geometry is structured, forcing isotropy carries measurable costs. Ablations confirm symplectic coupling drives the gains. Relevant for self-supervised learning practitioners exploring beyond contrastive methods.