Flama Ships an ASGI Framework That Serves REST, ML Models and LLMs Through One Rust-Accelerated Core

arXiv 2608.18733·medium signalXBlueskyLinkedInCopy link

Flama (arXiv 2608.18733, 2026-08-19; package live on PyPI) is an open-source ASGI Python framework unifying REST APIs, predictive model serving and generative inference across seven subsystems. Notable pieces for builders: a .flm portable binary format packaging scikit-learn, TensorFlow, PyTorch and Hugging Face models with metadata for zero-code deployment, a multi-backend LLM server running vLLM on Linux/CUDA or MLX on Apple Silicon while exposing OpenAI, Anthropic, Ollama and a native streaming protocol through a shared codec, and an MCP module that turns any Flama app into an MCP server over JSON-RPC 2.0. Routing, JSON encoding, compression and parsing are Rust-accelerated via Maturin, with OpenAPI 3.2.0 generated from handler signatures.