Text Simplification with Deep Neural Network Using Knowledge Transfer

Abstract

Text simplification aims to rephrase complex text into simpler text, where the text we are mainly considering is the English text sentences. Transfer learning from pre-trained text embeddings and models has recently shown great success on a range of natural language processing tasks and is, therefore, a focus method for our work. This thesis’s first focus is to avoid using parallel corpus with sentence pairs. We propose an unsupervised method to overcome the need for parallel data and similarity constraint loss for preserving the original meaning. Moreover, an asymmetric denoising technique is adopted better to learn various features from sentences with different complexity. The results demonstrate that the denoising method can improve the performance, and the content similarity constraint can help preserve the content in our unsupervised method. The second focus of this thesis is to define a novel approach to refining the existing noisy parallel datasets available for text simplification. After refining the dataset, our approach involves fine-tuning a pre-trained language model with a new proposed tuning strategy and decoding with a task-specific strategy. Our data refining method can generate a better dataset for the text simplification task, and the proposed fine-tuning strategy will accelerate model convergence. Moreover, the decoding strategy can greatly improve the model’s performance. The third focus of this thesis is to propose a prompting-based method without model fine-tuning. The proposed method transfers the text simplification task to the text denoising task with adaptive prompts. Our decoding vocabulary constraint technology also makes the output sentence simplicity controllable. The extensive experiments show that our proposed methodology can achieve state-of-the-art results considering many of the automatic evaluation metrics.

More information

Identifiers

ORCID for Kate Farrahi: orcid.org/0000-0001-6775-127X

Catalogue record

Export record