Reddit
Replacing a 3GB SQLite Database with a 10MB Finite State Transducer — 300x Size Reduction
A blog post trending on r/programming (194 upvotes) details replacing a 3GB SQLite database with a 10MB FST binary for a Finnish language dictionary application. FSTs compress both prefixes and suffixes (unlike tries which only share prefixes), making them especially effective for agglutinative languages where a small set of suffixes repeats frequently. The BurntSushi/fst Rust library powers the implementation. For builders working with large lookup tables or dictionary-style data, FSTs offer dramatic space savings with near-instant prefix/fuzzy search.
Source
↳ Follow the thread