For instance, Galactica can recommend possible citations for scientific concepts by mimicking existing citation patterns (“ Q: Is there any research on the effect of climate change on the great barrier reef? A: Try the paper ‘ Global warming transforms coral reef assemblages’ by Hughes, et al. Machine-learning systems infamously exacerbate existing societal biases, and Galactica is no exception. Indeed, an influential paper highlighting these risks prompted Google to fire one of the paper’s authors in 2020, and eventually disband its AI ethics team altogether. The risks associated with large language models are well understood. This reflects the biases lurking in the model’s training data, and Meta’s apparent failure to apply appropriate checks around the responsible AI research. Other critics reported that Galactica, like other language models trained on data from the internet, has a tendency to spit out toxic hate speech while unreflectively censoring politically inflected queries. ![]() However, peer reviewers at academic journals and conferences are already time-poor, and this could make it harder than ever to weed out fake science. This is to say nothing of exacerbating existing concerns about students using AI systems for plagiarism.įake scientific papers are nothing new. Galactica could make it easier for bad actors to mass-produce fake, fraudulent or plagiarised scientific papers. ![]() At worst, it risks further eroding public trust in scientific research. If a user already needs to be a subject matter expert in order to check the accuracy of Galactica’s “summaries”, then it has no use as an explanatory tool.Īt best, it could provide a fancy autocomplete for people who are already fully competent in the area they’re writing about. In practice, Galactica was enabling the generation of misinformation – and this is dangerous precisely because it deploys the tone and structure of authoritative scientific information. We found it would use all the right buzzwords, but get the actual details wrong – for example, mixing up the details of related but different algorithms. We also asked Galactica to explain technical concepts from our own fields of research. However, users quickly noticed that, while the explanations it generates sound authoritative, they are often subtly incorrect, biased, or just plain wrong. Galactica’s press release promoted its ability to explain technical scientific papers using general language. Authoritative, but subtly wrong bullshit generator Not only did Galactica reproduce many of the problems of bias and toxicity we have seen in other language models, it also specialised in producing authoritative-sounding scientific nonsense. However, once Galactica was opened up for public experimentation, a deluge of criticism followed. Galactica apparently outperforms other models at problems like reciting famous equations (“ Q: What is Albert Einstein’s famous mass-energy equivalence formula? A: E=mc²”), or predicting the products of chemical reactions (“ Q: When sulfuric acid reacts with sodium chloride, what does it produce? A: NaHSO₄ + HCl”). ![]() The preprint paper associated with the project (which is yet to undergo peer review) makes some impressive claims. Google's powerful AI spotlights a human cognitive glitch: Mistaking fluent speech for fluent thought The designers highlighted specialised scientific information like citations, maths, code, chemical structures, and the working-out steps for solving scientific problems. Galactica also used text from scientific papers uploaded to the (Meta-affiliated) website PapersWithCode. Most modern language models learn from text scraped from the internet. Galactica is a language model, a type of AI trained to respond to natural language by repeatedly playing a fill-the-blank word-guessing game.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |