Neural networks have been developed to detect generated inserts within texts.

Articles on the systems Papilusion and AIpom have been published in the digital archive of ACL Anthology. As language models like ChatGPT and GigaChat gain popularity and usage, it becomes increasingly challenging to distinguish between original human-written text and generated content. Scientific publications and theses are already being authored with the help of artificial intelligence. Therefore, it is crucial to develop tools that can identify AI-generated inserts in texts. A team of researchers from HSE University has proposed their solutions to this problem at the international scientific competitions SemEval 2024 and DAGPap24.

The AIpom model was used to identify the boundaries between original and generated fragments in scientific articles. The ratio of machine-generated text to author-written text varied in each paper. For model training, the organizers provided texts on a single topic, but during the validation phase, the topics changed, complicating the task.

“Models perform well with familiar topics, but when faced with new themes, the results decline,” says one of the article's authors, research intern at the Scientific and Educational Laboratory of Models and Methods of Computational Pragmatics at the Faculty of Computer Science at HSE University Alexander Shirnin. “It’s like a student who, having learned to solve one type of problem, struggles to solve a question on an unfamiliar topic or from a different subject.”

To enhance the system's efficiency, the researchers decided to combine two models—a decoder and an encoder. In the first stage, a decoder neural network was used, which received an instruction plus the original text as input and produced a fragment of text believed to be generated by AI. Then, in the original text, a segment was marked with the label <BREAK> to indicate where the model predicted the generated fragment began. The encoder worked with the text labeled in the first stage and refined the decoder's predictions by classifying each token—the smallest unit of text, either a word or a part of a word—and indicating whether it was written by a human or AI. This approach improved accuracy compared to systems that used only one type of model: AIpom secured 2nd place in the SemEval-2024 scientific competition.

The Papilusion model also distinguished between human-written text and generated content. It categorized text segments into four categories: written by a human, corrected using synonyms, generated by a model, and briefly paraphrased. The task was to accurately identify each category. The number of categories and the length of inserts in the texts varied.

In this case, the developers used three models, all of the same type—encoders. They were trained to predict one of four categories for each token in the text, with all models being trained independently. When a model made an error, it was penalized and retrained while freezing the lower layers of the model.

“Each model has a different number of layers depending on its architecture. When we train a model, we might leave the first ten layers untouched and only adjust the last two. This is done to ensure that we don’t lose important data embedded in the initial layers during training,” explains Alexander Shirnin. “You can compare this to an athlete who makes a mistake in their arm movement. We need to explain just that to them, rather than erase their knowledge and retrain them from scratch, as that might cause them to forget how to move correctly overall. The logic is similar here. The method isn’t universal and might not be effective for some models, but in our case, it worked.”

Three encoders independently determined the category for each token (word). The final system selection was based on which category received the majority of votes. In the competition, the Papilusion system placed 6th out of 30.

As the researchers note, current models for identifying AI functions well but still have limitations, primarily in poorly handling data that falls outside the training set, and there is a general lack of diverse data for training the models.

“To gather more data, we need to focus on its collection. Both companies and laboratories are working on this. Specifically for such tasks, datasets need to be compiled where multiple AI models and correction methods are employed in the texts,” comments the researcher. “That is, rather than simply continuing text with one model, we should create more realistic situations: at times, ask a model to complete a text, rewrite the beginning to make it fit better, delete parts of it, or try to generate a section in a new style using a different prompt (instruction) for the model. Additionally, it’s crucial to gather data in other languages and across various topics.”