Are we ignoring "Data Entropy" in the race for massive Context Windows? (Plus a tool I built to test this)
Hi everyone, There’s a massive trend right now towards "Infinite Context". The marketing pitch is: "Just dump your entire knowledge base into the prompt, the model will figure it out." I think this is a dangerous trap. From my exper…