Why AI Companies Lied About Context Windows
Channel: Tiago Forte
Uploaded:
Duration: PT584S
Read a transcript preview of “Why AI Companies Lied About Context Windows” from Tiago Forte, uploaded on 2025-10-28.
Transcript preview
Language: English (original, YouTube automatic captions)
[0:00] Consider this. You're uploading multiple
[0:02] documents into Chat GPT, crafting
[0:04] detailed prompts, adding tons of project
[0:07] files, all to give it as much context as
[0:09] possible. And yet, somehow, it keeps
[0:11] missing the most important details.
[0:17] If you're experiencing this as well, you
[0:19] might assume that you need to give the
[0:20] AI better instructions. Or maybe you
[0:22] think you have to give it even more
[0:24] context. But what I've learned through
[0:25] weeks of testing is that this is a
[0:27] systemic, widespread issue. In the next
[0:30] few minutes, we'll explore the research
[0:32] that reveals the real performance limits
[0:34] of AI. And finally, I'll walk you
[0:35] through the approach that I've developed
[0:37] to work with these limitations rather
[0:40] than against them. You know these
[0:41] impressive context windows that AI
[0:43] companies love to promote? 200,000
[0:45] tokens, 1 million tokens. A recent study
[0:48] of 18 leading LLMs tested their
[0:51] performance across different context
[0:53] lengths. And the results tell a story
[0:55] that's very different from the marketing
[0:57] materials. So, when Chat GPT and Claude
[1:00] have promised 200,000 tokens, the
[1:02] effective context window, where you get
[1:04] reliable results, is actually closer to
[1:07] between 50 and 100,000 tokens. In other
[1:11] words, in every case, you're actually
[1:13] getting somewhere between 25 and 50% of
[1:17] the advertised capacity in terms of
[1:19] dependable performance from major LLMs.
[1:22] This is a systemic, widespread issue,
[1:24] and it already has a name: context rot.
[1:28] Now, this isn't necessarily deceptive
[1:30] ma