Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-31 · high
Each page gets its own database row and vector set, so a single edited page no longer forces a full site re-index.
Why now: Agno posted the 3.0.3 release notes on GitHub on August 30, 2026, one day before this coverage.
Agno's 3.0.3 release changes how the framework stores crawled websites for retrieval. That matters for anyone running ingestion at scale, where re-embedding an entire site because one page changed burns real API budget and time. Each page gets its own content row, and that row's id matches the content_id carried by its vectors. Pages get listed, refreshed, or deleted one at a time, while the parent site row aggregates status across all of them.
The payoff shows up on re-crawl. Unchanged pages skip embedding entirely. Changed pages get only their own vectors replaced. Pages that dropped out of the sitemap get pruned instead of lingering as stale entries.
The release also adds a SitemapReader that finds pages by checking robots.txt for Sitemap: lines, falling back to /sitemap.xml, and handling gzip files and nested sitemap indexes.
None of this is specific to Agno. The mechanism is per-page rows, a digest check per page, and incremental writes instead of a full teardown and rebuild. The release notes don't say what digest algorithm it uses, or how it handles a page whose content changes without its URL changing, so anyone copying the pattern still has to make that call.
Each link below shares sources, entities, or timing with this story.
Agno supports MCP / Shared entity: Agno / Same source domain / Earlier coverage
Linked by a graph relationship (Agno supports MCP); both cover Agno; reported by the same outlet (github.com).
Linked by a graph relationship (Agno supports MCP); both cover Agno; reported by the same outlet (github.com).
Agno uses IPython / Shared entity: Agno / Same source domain / Earlier coverage
Linked by a graph relationship (Agno uses IPython); both cover Agno; reported by the same outlet (github.com).
Linked by a graph relationship (Agno uses IPython); both cover Agno; reported by the same outlet (github.com).
Agno supports MCP / Shared entity: Agno / Same source domain / Earlier coverage
Linked by a graph relationship (Agno supports MCP); both cover Agno; reported by the same outlet (github.com).
Agno supports MCP / Same source domain / Shared topic / Tension
Linked by a graph relationship (Agno supports MCP); reported by the same outlet (github.com); overlapping topics (changed, copy).
Agno supports MCP / Same source domain
Linked by a graph relationship (Agno supports MCP); reported by the same outlet (github.com).
Linked by a graph relationship (Agno supports MCP); reported by the same outlet (github.com).