I'm sorry but the Google 10 Million context and 1 million context marketing looks like they're at it again.
Here is some information to help explain why I am thinking about this. A post related to this issue - https://www.reddit.com/r/ChatGPT/comments/1at332h/bill_french_on_linkedin_gemini_has_a_memory/
leads you to a linked in blog post here
And article here
https://www.linkedin.com/pulse/gemini-has-memory-feature-too-bill-french-g0igc/
The article goes on to explain how Google is doing "memory" Blog post entitled Gemini has a memory feature too. And again the feature is related to a form of RAG than it is related to any technological advancement.
Michael Boyens replies with this question:
Great insights into use of Google docs for context when prompting. Not sure how this equivalent to memory feature with ChatGPT which uses both context and prompts across all chat threads though?
It's a fair question and it's my same question. Are they calling RAG = Context?
I knew 10 million tokens sounded suspicious. What's irking is that my initial reaction to Gemini pro the last time I reviewed it was that it seemed like the search guys are really trying to weave "things that come from legacy search" into what they are attempting to call "AI". When in fact, it's literal upgrades to search.
I0 million token context can't be real. In fact, I don't want it to be real. It has no practical purpose (unless it was actually real) other than getting poor prompters/Data Scientists shoving in corpus of text and then running the LLM and saying see it's not magic; see it doesn't work.
The notion that you can roll a frame of context up to 10 million tokens with pure coherence can't be currently possible. I can't possibly believe that. Not without a quantum computer or 1 billion Grace Hopper GPU's. The idea seems ridiculous to me.
RAG is awesome but just call it RAG or A* or search or something. Don't say context. Context is about the coherence of the conversation. The ability to "know" what I am saying or referring to without me having to remind you.
I also respect Google and Microsoft for thinking about how to pre-accomplish RAG for folks with low code solutions because in general many people aren't great at it. I get that. But it's not the evolution of this technology. If you do that and market it like that then people will always have disappointment on their minds because "they can't get the damned thing to work."
The most innovative and coolest things I have built have been based on a lot of data clean up, annotations, embeddings and RAG.
The technology needs innovation and I respect Google for pushing and wanting to get back into the game but don't try to tomfoolery us. How many times are you going to keep doing these types of marketing things before people just outright reject your product.
Context, for all intents and purposes, works as a 1-shot mechanism. I need to know that I can depend on your context window length for my work and conversation.
If I give you a million lines of code I don't want to simply search through my code base. I want you to understand the full code base in it's complete coherence. That is the only way you would be able to achieve architectural design and understanding.
We all obviously deal with this today when having conversations with GPT. There is a point in the conversation where you realize GPT lost the context window and you have to scroll up, grab a piece of code or data and "remind" GPT what it is you guys are talking about.
It's just something we all deal with and inherently understand. At least I hope you do.
Coherence is the magic in these models. It's the way your able to have a conversation with GPT like it's a human speaking to you. I even have arguments with GPT and it is damn good at holding it's ground many times. Even getting me to better understand it's points. There are times I have gone back to GPT and said DAMN you're right I should have listened the first time. It's weird. It's crazy. Anyways, point is this:
RAG IS NOT CONTEXT; RAG IS NOT COHERENCE; RAG IS NOT MEMORY.
Do better. I am glad there is competition so I am rooting for you Google.
[link] [comments]