This is a news story, published by Quanta Magazine, that relates primarily to Meta news.
For more Ai research news, you can click here:
more Ai research newsFor more news from Quanta Magazine, you can click here:
more news from Quanta MagazineOtherweb, Inc is a public benefit corporation, dedicated to improving the quality of news people consume. We are non-partisan, junk-free, and ad-free. We use artificial intelligence (AI) to remove junk from your news feed, and allow you to select the best tech news, business news, entertainment news, and much more. If you like this article about Ai research, you might also like this article about
word embeddings. We are dedicated to bringing you the highest-quality news, junk-free and ad-free, about your favorite topics. Please come every day to read the latest language models news, embeddings news, news about Ai research, and other high-quality news about any topic that interests you. We are working hard to create the best news aggregator on the web, and to put you in control of your news feed - whether you choose to read the latest news through our website, our news app, or our daily newsletter - all free!
large language modelsQuanta Magazine
•
70% Informative
In Meta ’s open-source Llama 3 model, each word contains 4,096 numbers; for GPT-3 , it's 12,288 .
In concert, they encode mathematical relationships between words that can look surprisingly like meaning.
The downside of machine-learned embeddings is that many descriptions encoded in each list of numbers are not interpretable by humans.
Embeddings — contextual coordinates, based on statistics — are how an LLM can find a good starting point for making its next-word predictions.
Certain words in certain contexts fit together better than others, sometimes so precisely that literally no other words will do.
When these mathematical objects fit together in a way that coincides with our expectations, it feels like intelligence; when they don't, we call it a “hallucination.
VR Score
78
Informative language
85
Neutral language
35
Article tone
informal
Language
English
Language complexity
48
Offensive language
not offensive
Hate speech
not hateful
Attention-grabbing headline
detected
Known propaganda techniques
not detected
Time-value
long-living
External references
1
Source diversity
1
Affiliate links
no affiliate links