Microsoft LongNet: One BILLION Tokens LLM — David Shapiro ~ AI (06.07.2023)
Microsoft LongNet: One BILLION Tokens LLM — David Shapiro ~ AI (06.07.2023)
We could have AI models in a couple years that hold the entire internet in their context window.
8 comments
-
We could have AI models in a couple years that hold the entire internet in their context window.
That's a really bold claim.
-
Also not sure how that would be helpful. If every prompt needs to rip through those tokens first, before predicting a response, it'll be stupid slow. Even now with llama.cpp, it's annoying when it pauses to do the context window shuffle thing.
-
-
Anyone know what progress has been made with hallucinations .
-
Perplexity has pretty much solved that since it searches the internet and uses the information it finds. But I don't know about any advances to solve it directly in LLMs.
-
8 comments