EsoLang-Bench: Using Obscure Programming Languages to Test Whether LLMs Actually Reason

EsoLang-Bench·medium signal

EsoLang-Bench (90pts, 49 comments on HN) evaluates LLMs on esoteric programming languages specifically designed to prevent memorization, isolating genuine reasoning from training-data recall. The benchmark directly challenges the validity of HumanEval, SWE-bench, and similar coding leaderboards where models may be pattern-matching memorized solutions rather than reasoning. This surfaces as a practitioner-validated extension of the benchmark-validity critique that has been building in the HN community over the past two weeks.