Excited to share Arch-Router, our research and model for LLM routing. Routing to the right LLM is still an elusive problem, riddled with nuance and blindspots. For example: “Embedding-based” (or simple intent-classifier) routers sound good on pap…

Pretty big release milestone for our open source AI-native proxy server project. This one’s based on real-world feedback from deployments (at T-Mobile) and early design work with Box. Originally, the proxy server offered a low-latency universal …

The agent frameworks we have today (like LangChain, LLamaIndex, etc) are helpful but implement a lot of the core infrastructure patterns in the framework itself – mixing concerns between the low-level work and business logic of agents. I think this bec…

The agent frameworks we have today (like LangChain, LLamaIndex, etc) are helpful but implement a lot of the core infrastructure patterns in the framework itself – mixing concerns between the low-level work and business logic of agents. I think this bec…

I am thrilled about our latest release: Arch 0.2.8. Initially we handled calls made to LLMs – to unify key management, track spending consistently, improve resiliency and improve model choice – but we just added support for an ingress listener (o…

In the race for ever-larger models, its easy to forget just how powerful small LLMs can be—blazingly fast, resource-efficient, and surprisingly capable. I am biased, because my team builds these small open source LLMs – but the potential to creat…

I naturally post about models (have a bunch on HF) over tools in this sub, but I also use tools and LLMs to develop agentic systems, and find that there is this mad rush to use the latest agentic framework as if that's going to magically accelerate…