The University of Southampton
University of Southampton Institutional Repository

Exploring sentence embeddings for better argument relation classification

Exploring sentence embeddings for better argument relation classification
Exploring sentence embeddings for better argument relation classification
This thesis investigates methods to improve argument relation classification by augmenting sentence embeddings and leveraging Large Language Models (LLMs). Argument relation classification, a subtask of argument mining, involves predicting relationships, such as support or attack, between argumentative components in unstructured text. While sentence embeddings have demonstrated state-of-the-art performance in standard text classification, little research has examined their potential for argument relation classification. To address this gap, we evaluate three sentence embedding models—InferSent, Sent2Vec, and SBERT—on text classification tasks. Our findings show that SBERT performs best, underscoring the importance of Transformer-based architectures in capturing sentence semantics. However, as SBERT relies on annotated data, we explore self-supervised fine-tuning of BERT with contrastive learning and various data augmentation strategies to create domain-adaptive sentence embeddings. These embeddings outperform SBERT on text classification tasks when pre-trained and fine-tuned specifically for the domain. Building on this approach, we apply domainadaptive BERT to argument relation classification, achieving superior results over baseline models. Additionally, we investigate the use of LLMs for argument relation classification, combining sentence embeddings with LLMs using direct and Chain-of-Thought prompting strategies. Fine-tuned Llama2 models with Chain-of-Thought prompting outperform baseline methods, demonstrating the value of LLMs in capturing complex argumentative structures across multi-sentence inputs. This work contributes to argument mining by advancing the use of sentence embeddings and LLMs in argument relation classification.
University of Southampton
Zhu, Jiatong
52569115-5d72-4fc0-8876-a66b991ed209
Zhu, Jiatong
52569115-5d72-4fc0-8876-a66b991ed209
Middleton, Stuart
404b62ba-d77e-476b-9775-32645b04473f
Norman, Tim
663e522f-807c-4569-9201-dc141c8eb50d

Zhu, Jiatong (2025) Exploring sentence embeddings for better argument relation classification. University of Southampton, Doctoral Thesis, 122pp.

Record type: Thesis (Doctoral)

Abstract

This thesis investigates methods to improve argument relation classification by augmenting sentence embeddings and leveraging Large Language Models (LLMs). Argument relation classification, a subtask of argument mining, involves predicting relationships, such as support or attack, between argumentative components in unstructured text. While sentence embeddings have demonstrated state-of-the-art performance in standard text classification, little research has examined their potential for argument relation classification. To address this gap, we evaluate three sentence embedding models—InferSent, Sent2Vec, and SBERT—on text classification tasks. Our findings show that SBERT performs best, underscoring the importance of Transformer-based architectures in capturing sentence semantics. However, as SBERT relies on annotated data, we explore self-supervised fine-tuning of BERT with contrastive learning and various data augmentation strategies to create domain-adaptive sentence embeddings. These embeddings outperform SBERT on text classification tasks when pre-trained and fine-tuned specifically for the domain. Building on this approach, we apply domainadaptive BERT to argument relation classification, achieving superior results over baseline models. Additionally, we investigate the use of LLMs for argument relation classification, combining sentence embeddings with LLMs using direct and Chain-of-Thought prompting strategies. Fine-tuned Llama2 models with Chain-of-Thought prompting outperform baseline methods, demonstrating the value of LLMs in capturing complex argumentative structures across multi-sentence inputs. This work contributes to argument mining by advancing the use of sentence embeddings and LLMs in argument relation classification.

Text
thesis_pdf:a - Version of Record
Available under License University of Southampton Thesis Licence.
Download (2MB)
Text
Final-thesis-submission-Examination-Miss-Jiatong-Zhu
Restricted to Repository staff only

More information

Submitted date: 12 May 2025
Published date: May 2025

Identifiers

Local EPrints ID: 500917
URI: http://eprints.soton.ac.uk/id/eprint/500917
PURE UUID: 44eb7c21-80a3-4d42-bff7-4fdc2e6ea297
ORCID for Jiatong Zhu: ORCID iD orcid.org/0009-0003-9060-9003
ORCID for Stuart Middleton: ORCID iD orcid.org/0000-0001-8305-8176
ORCID for Tim Norman: ORCID iD orcid.org/0000-0002-6387-4034

Catalogue record

Date deposited: 16 May 2025 16:35
Last modified: 11 Sep 2025 03:18

Export record

Contributors

Author: Jiatong Zhu ORCID iD
Thesis advisor: Stuart Middleton ORCID iD
Thesis advisor: Tim Norman ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×