microsoft/Foundry-Local: On-Device AI Inference SDK With Zero Network Latency at 2.1K Stars

GitHub·medium signal

Microsoft's Foundry-Local embeds generative AI directly into apps with all inference on-device — data never leaves the device. Single SDK handles text generation and Whisper speech-to-text with automatic hardware acceleration (NPU > GPU > CPU). SDKs for JavaScript, C#, and Python. Since February 2026, supports combination with GitHub Copilot SDK for fully autonomous coding workflows on local hardware. Large multimodal models now run in disconnected sovereign environments.