Exploring sentence embeddings for better argument relation classification
Exploring sentence embeddings for better argument relation classification
This thesis investigates methods to improve argument relation classification by augmenting sentence embeddings and leveraging Large Language Models (LLMs). Argument relation classification, a subtask of argument mining, involves predicting relationships, such as support or attack, between argumentative components in unstructured text. While sentence embeddings have demonstrated state-of-the-art performance in standard text classification, little research has examined their potential for argument relation classification. To address this gap, we evaluate three sentence embedding models—InferSent, Sent2Vec, and SBERT—on text classification tasks. Our findings show that SBERT performs best, underscoring the importance of Transformer-based architectures in capturing sentence semantics. However, as SBERT relies on annotated data, we explore self-supervised fine-tuning of BERT with contrastive learning and various data augmentation strategies to create domain-adaptive sentence embeddings. These embeddings outperform SBERT on text classification tasks when pre-trained and fine-tuned specifically for the domain. Building on this approach, we apply domainadaptive BERT to argument relation classification, achieving superior results over baseline models. Additionally, we investigate the use of LLMs for argument relation classification, combining sentence embeddings with LLMs using direct and Chain-of-Thought prompting strategies. Fine-tuned Llama2 models with Chain-of-Thought prompting outperform baseline methods, demonstrating the value of LLMs in capturing complex argumentative structures across multi-sentence inputs. This work contributes to argument mining by advancing the use of sentence embeddings and LLMs in argument relation classification.
University of Southampton
Zhu, Jiatong
52569115-5d72-4fc0-8876-a66b991ed209
May 2025
Zhu, Jiatong
52569115-5d72-4fc0-8876-a66b991ed209
Middleton, Stuart
404b62ba-d77e-476b-9775-32645b04473f
Norman, Tim
663e522f-807c-4569-9201-dc141c8eb50d
Zhu, Jiatong
(2025)
Exploring sentence embeddings for better argument relation classification.
University of Southampton, Doctoral Thesis, 122pp.
Record type:
Thesis
(Doctoral)
Abstract
This thesis investigates methods to improve argument relation classification by augmenting sentence embeddings and leveraging Large Language Models (LLMs). Argument relation classification, a subtask of argument mining, involves predicting relationships, such as support or attack, between argumentative components in unstructured text. While sentence embeddings have demonstrated state-of-the-art performance in standard text classification, little research has examined their potential for argument relation classification. To address this gap, we evaluate three sentence embedding models—InferSent, Sent2Vec, and SBERT—on text classification tasks. Our findings show that SBERT performs best, underscoring the importance of Transformer-based architectures in capturing sentence semantics. However, as SBERT relies on annotated data, we explore self-supervised fine-tuning of BERT with contrastive learning and various data augmentation strategies to create domain-adaptive sentence embeddings. These embeddings outperform SBERT on text classification tasks when pre-trained and fine-tuned specifically for the domain. Building on this approach, we apply domainadaptive BERT to argument relation classification, achieving superior results over baseline models. Additionally, we investigate the use of LLMs for argument relation classification, combining sentence embeddings with LLMs using direct and Chain-of-Thought prompting strategies. Fine-tuned Llama2 models with Chain-of-Thought prompting outperform baseline methods, demonstrating the value of LLMs in capturing complex argumentative structures across multi-sentence inputs. This work contributes to argument mining by advancing the use of sentence embeddings and LLMs in argument relation classification.
Text
thesis_pdf:a
- Version of Record
Text
Final-thesis-submission-Examination-Miss-Jiatong-Zhu
Restricted to Repository staff only
More information
Submitted date: 12 May 2025
Published date: May 2025
Identifiers
Local EPrints ID: 500917
URI: http://eprints.soton.ac.uk/id/eprint/500917
PURE UUID: 44eb7c21-80a3-4d42-bff7-4fdc2e6ea297
Catalogue record
Date deposited: 16 May 2025 16:35
Last modified: 11 Sep 2025 03:18
Export record
Contributors
Author:
Jiatong Zhu
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics