How much longer will RAG be useful in light of Gemini’s 1M and 10M token context window and 99% accuracy?
The Gemini release was really interesting in that they sort of buried the lede by not mentioning the 99% accuracy of the context window. The 128k context window of OpenAI will fall down pretty quickly and really is only 32k-64k if you care about your c…