Microsoft Foundry-Local: Unified On-Device AI Runtime Replacing Ollama and llama.cpp

GitHub·high signal

Microsoft open-sourced Foundry-Local, a unified local AI runtime SDK that replaces the need for separate tools like whisper.cpp, llama.cpp, or Ollama. One SDK handles both chat and audio with automatic hardware acceleration (NPU > GPU > CPU), ships self-contained with no external dependencies, and integrates with Azure AI Foundry for governance. Available for JavaScript, C#, and Python with support for CPUs, NVIDIA GPUs, and NPUs on Copilot+ PCs. Already at 2,079 GitHub stars.