Science & Technology

I was right again

by The PrimeTime

Share:

📚 Main topics

  • Images as a more efficient model inputThe video discusses DeepSeek research suggesting that one image token can represent information equivalent to about 10 text tokens, and a separate claim that using images could reduce end-to-end costs. 0:35
  • Why images may work wellImage embeddings can represent a wider range of information than discrete text tokens, potentially giving models more context in fewer tokens. 3:11
  • Testing the idea on a gameThe creator compared text and image versions of a game’s context file by letting models play a tower-defense game repeatedly. 6:48

✨ Key takeaways

  • Results were mixedThe text-context setup completed more games and played much faster, while the image-context setup took considerably longer but achieved similar damage results. 8:21
  • Token totals need contextThe image-context run used fewer tokens, but it also completed fewer than half as many games, making a direct comparison difficult. 10:57

🧠 Lessons learned

  • Benchmarks depend on the taskThe experiment suggests that image-based context may be harder or more confusing for this particular game, despite possible token savings. 11:27
  • Test claims for yourselfOnline claims about optimizations are worth investigating, but developers should run their own experiments rather than assuming a technique will work in every situation. 11:58

🏁 Conclusion/next steps

  • Image workflows may improveTools and infrastructure are largely designed around text, so image-based approaches could become more effective as caching and other support improve. 12:29
  • Keep experimentingTreat text-versus-image comparisons as task-dependent, and look for opportunities to test potential optimizations in your own work. 13:00

Transcript excerpt

0:04 You see that face? You know what that face is? That that is the smug face of somebody who was proven correct. And the best part, it's right after caught in 8K. You're even told by your friends you're wrong. So, you're probably thinking, "Okay, what what are we talking about? What are you even trying to say? What were you proven right about?" Well, my friends, it turns out after many hurt feelings and being told I was wrong, images might actually be better for models than text. I knew it.

0:35 Don't make it do image recognition on that, dude. >> No, no, you just do this. What are you talking about? >> Hey, >> now this isn't just me saying this, of course. This is actually DeepSeek releasing a new paper that suggests that if you provide images, you could pull out 10 text token from a single image token with near 100% accuracy. In other words, a model's internal representation of an image is 10 times efficient as its internal representation of text.

🔒 The full, searchable transcript is available with Pro.

🔒 Unlock Premium Features

This is a premium feature. Upgrade to unlock unlimited Q&A, transcripts, mindmaps, and translations.

Questions & Answers

Common questions about this video

What claim did the DeepSeek paper make about using images instead of text?

It suggested that a model could recover about 10 text tokens from a single image token with near-100% accuracy, making an image representation more efficient for that information. 0:35

Why might image tokens represent information more efficiently than text tokens?

Text tokens are discrete and come from a finite set, while image-token embeddings can vary continuously. That gives an image token more room to express information than a text token. 3:42

What experiment did the creator run to compare text and image context?

He had models play his tower defense game using either the game's context as text or the same context packed into an image, then compared their performance over repeated games. 6:48

How did the models perform with the text context?

They completed 50 successful games, taking about nine minutes per game on average, and often placed towers and cards in an effective order to maximize damage. 8:21

How did the image-context experiment compare with the text-context experiment?

The image-context run completed only 23 successful games and took about 20 minutes per game on average, with some taking around 30 minutes. Damage was broadly similar, but the image-context games were much slower. 9:24

What conclusion did the creator draw from his experiment?

He concluded that using an image for context seemed harder and more confusing in this test, despite using fewer tokens. He cautioned that the results were specific to his experiment and that people should test the approach themselves. 10:57

Does the creator recommend converting all text to images to save tokens?

No. He says not to rush into converting everything based on online claims; the value depends on the task, and the technology may not yet be optimized for image inputs. 11:58

🔒 Unlock Premium Features

Access to Chat is a premium feature. Upgrade now to unlock unlimited studying tools.

🔒 Unlock Premium Features

Access to Mindmap is a premium feature. Upgrade now to unlock unlimited studying tools.

🔒 Unlock Premium Features

Access to Translation is a premium feature. Upgrade now to unlock unlimited studying tools.

Get unlimited summaries, Q&A, transcripts and more with Pro

Upgrade to Pro

Suggestions

🔒 Unlock Premium Features

Access to AI Suggestions is a premium feature. Upgrade now to unlock unlimited studying tools.