ASIC Critiques Large Language Models

This is a Australia news story, published by Ars Technica, that relates primarily to Gen AI news.

Australia news

For more Australia news, you can click here:

more Australia news

Gen AI news

For more Gen AI news, you can click here:

more Gen AI news

tech industry trends news

For more tech industry trends news, you can click here:

more tech industry trends news

Ars Technica news

For more news from Ars Technica, you can click here:

more news from Ars Technica

About the Otherweb

Otherweb, Inc is a public benefit corporation, dedicated to improving the quality of news people consume. We are non-partisan, junk-free, and ad-free. We use artificial intelligence (AI) to remove junk from your news feed, and allow you to select the best tech news, business news, entertainment news, and much more. If you like tech industry trends news, you might also like this article about

lengthy documents

. We are dedicated to bringing you the highest-quality news, junk-free and ad-free, about your favorite topics. Please come every day to read the latest larger documents news, AI summaries news, tech industry trends news, and other high-quality news about any topic that interests you. We are working hard to create the best news aggregator on the web, and to put you in control of your news feed - whether you choose to read the latest news through our website, our news app, or our daily newsletter - all free!

ASIC references

Ars Technica

•

Australian government trial finds AI is much worse than humans at summarizing

Summary

Nutrition label

89% Informative

Australia 's Securities and Investments Commission evaluated large language models' ability to quickly summarize lengthy documents for easier human consumption.

The Llama2-70B model was judged as significantly worse than those provided by humans.

The study has a number of limitations that make it hard to generalize about the summarizing capabilities of state-of-the-art models in the present day .

Larger models with bigger context windows and better embedding strategies may have more success, the authors write, because "finding references in larger documents is a notoriously hard task for LLMs." Despite the results, ASIC says it still believes "there are opportunities for Gen AI as the technology continues to advance... Technology is advancing in this area and it is likely that future models will improve performance and accuracy of results.".

VR Score

Informative language

Neutral language

Article tone

formal

Language

English

Language complexity

Offensive language

not offensive

Hate speech

not hateful