The SignalToday's digest →

Did an AI or a human write that paper? New detector decides by examining its ideas

Science · original · full text captured 17 November 2026 · WhatsApp · Email

Opens in a new window Opens an external website Opens an external website in a new window

This website utilizes technologies such as cookies to enable essential site functionality, as well as for analytics, personalization, and targeted advertising. Privacy Policy

Main content starts here

ScienceInsider

Technology

Did an AI or a human write that paper? New detector decides by examining its ideas

Algorithm looks for clues about a paper’s origin that text-based detectors miss

12 Oct 2026

2:05 AM ET

!Illustration of a stack of papers with green streams of 0s and 1s glowing upward from portions of the paper

Many journal editors and conference managers are drawing a line: Using artificial intelligence to polish a paper’s text is OK, if disclosed, but producing the entire paper with AI is not.A. Mastin/Science

More and more scientists use artificial intelligence (AI) to help write research papers, and journal editors and conference managers, faced with ballooning submissions, are drawing a line: Using AI to polish a paper’s text is OK, if disclosed, but producing the entire paper with AI is not. To enforce that rule, some gatekeepers are running submissions through AI-text detectors, but these tools can make mistakes—for example, labeling entire papers as computer-generated when authors only used AI to lightly edit the product.

Last week, researchers unveiled a novel approach to detecting AI provenance that they say largely avoids these shortcomings. Called IdeaLens, it focuses on whether a paper’s ideas—and not just the prose—came from a human or a machine.

“This is a completely new kind of detector and certainly represents an advance in the field, as it’s a novel idea executed well,” says Max Spero, CEO of Pangram, which offers one of the major commercial AI text detectors.

In essence, IdeaLens works by converting a document into a detailed outline of its main ideas, without using the original wording. IdeaLens then analyzes the structure and content of the ideas for telltale clues to determine whether they were likely produced by a human or a machine. In contrast, Pangram and other AI-text detectors only examine patterns in a document’s words and phrases.

In tests, IdeaLens produced verdicts notably different from current AI-text detectors. In one set of trials, for example, the researchers used large language models (LLMs) to create test documents based on outlines that contained ideas classified as generated by humans, and had Pangram and IdeaLens analyze the same set of documents. In the most stringent test using the most detailed outline, Pangram, analyzing just the documents’ text, flagged 92% of them as produced by AI, but IdeaLens found that AI had generated just 7%, the scientists report in a 5 October preprint. On 12 other tests of whether an AI made minor changes to a document, such as polishing, IdeaLens identified 95% of the human-generated ones as human whereas Pangram only flagged 44%, on average. At the same time, IdeaLens’s accuracy at flagging documents originally generated by AI rivaled that of Pangram and other current tools.

To create IdeaLens, developers trained the detector on more than 800,000 long-form documents of eight types, including academic, creative, and nonfiction writing. Each document had been classified by Pangram as mostly AI or human generated. Next, they used LLMs to generate an outline for each that identified a set of rhetorical elements particular to each type of document; in a research paper, for example, the outline identified its claim, argument, and evidence. Then AIs drew excerpts from the document that fit those elements and paraphrased this text, to avoid seeding the outline with words that IdeaLens might immediately identify as AI written. The team observed systematic differences between outlines that IdeaLens had predicted came from AI- or human-written texts. Human-based outlines, for example, provided more concrete details and evidence for claims.

Advertisement

The researchers acknowledge their study has limitations. For example, they didn’t independently determine whether the documents used to train IdeaLens were actually written by humans or AI, nor did they create their own data set of the two types for controlled testing—tasks that would have been too costly, says Rishanth Rajendhran, a Ph.D. student in computer science at the University of Maryland who is the study’s lead author. Creating that kind of training archive would be a good next step, he noted.

Another issue is that IdeaLens was not exposed to a type of hybrid document becoming common in science—a mix of ideas and writing produced by humans and machines. Such hybrids are produced, for example, when a scientist uses a chatbot to iteratively refine an idea or a manuscript. Detectors such as Pangram try to address such mixed-origin texts by telling users that a certain percentage of a document was likely generated by AI, but IdeaLens offers only a binary “AI or human” prediction.

Figuring out how to design tools like IdeaLens to accurately tell the difference will require more research, says Marijke Van Vlasselaer, a faculty member at the Free University of Brussels’s Solvay Business School who has studied AI-text detectors and oversees scholarly integrity at her institution. “The question of whose idea is it [human or AI] is rarely black and white,” she notes. “We humans, even with reflection, cannot always know what triggered an idea.”

IdeaLens may be picking up on some clues in the AI-generated outlines, such as proper nouns, that do not capture the essence of an idea, adds Misha Teplitskiy, a sociologist at the University of Michigan who studies innovation in science. “It’s very hard to know what IdeaLens is really classifying off of,” he says. The detector “will need more stress testing and validation before becoming a part of any AI policy,” Spero says.

The IdeaLens study didn’t attempt to answer the broader question of whether papers written mostly by humans are consistently better than ones from machines. But it is likely to add fuel to ongoing debates over how much AI-generated content is appropriate in scholarly communication. “Discerning the extent to which AI performs critical thinking—a task that, in our view, rightly belongs to human authors—is increasingly difficult,” the editors of The Journal of Clinical Psychiatry wrote in a 30 September editorial. But “100% AI-generated content seems irreconcilably problematic.”

Rajendhran says journals and conferences could use IdeaLens to monitor AI-written text, although flagging a pattern of such use is more informative than on a single manuscript, given that the software is not perfectly accurate. But he hopes researchers will use it to review their own papers before submitting them and reflect on what role AI played in shaping them. To that end, the team is sharing IdeaLens’s code for free, and anyone may run papers through the tool until the project’s resources to run the software are exhausted.

“We want to encourage people to use AI more intentionally for writing tasks,” he says. “There’s no replacement for human intuition and human insight.”

Science’s AI in Science reporting initiative is supported by Ray Rothrock & family.

doi: 10.1126/science.zt1x2j3

Relevant tags:

Technology

Artificial intelligenceScience’s AI in Science Reporting Initiative

About the author

Jeffrey Brainard

Author

Jeffrey Brainard is a reporter at Science in Washington, D.C., covering scientific publishing, open science, peer review, the science of science, and other topics. He can be reached on Signal at jbrainard.19 and on Bluesky at ‪@jeffreybrainard.bsky.social.

More from news

Can researchers bring order to the confusing world of AI ‘benchmarks’?

By

Ananya

To deter bioterrorism, U.S. imposes new rules on ordering synthetic DNA

By

Phie Jacobs

Researchers sue to block NSF from demolishing social science directorate

Support nonprofit science journalism

Amid the unprecedented upheaval in U.S. science, hard-hitting, trustworthy reporting has never been more important. As part of the AAAS mission, Science has built a global award-winning network of reporters and editors that independently cover the most important developments in research and policy.

Your tax-deductible contribution plays a critical role in sustaining this effort.

If, at any time, you would like to stop or make changes to your monthly gift, please contact the Philanthropy Office at philanthropy@aaas.org or 202-326-6636.

Support nonprofit science journalism

Help News from Science publish trustworthy, high-impact stories about research and the people who shape it. Please make a tax‑deductible gift today.