The University of Southampton
University of Southampton Institutional Repository

Feature Selection for Summarising: The Sunderland DUC 2004 Experience

Feature Selection for Summarising: The Sunderland DUC 2004 Experience
Feature Selection for Summarising: The Sunderland DUC 2004 Experience
In this paper we describe our participation in task 1-very short single-document summaries in DUC 2004. The task chosen is related to our research project, which aims to produce abstracting summaries to improve search engine result summaries. DUC allowed us to produce summaries no longer than 75 characters, therefore we focused on feature selection to produce a set of key words as summaries instead of complete sentences. Three descriptions of our summarisers are given. Each of the summarisers performs very differently in the six ROUGE metrics. One of our summarisers which uses a simple algorithm to produce summaries without any supervised learning or complicated NLP technique performs surprisingly well among different ROUGE evaluations. Finally we give an analysis of ROUGE and participants’ results. ROUGE is an automatic evaluation of summaries package, which uses n-gram matching to calculate the overlapping between machine and human summaries, and indeed saves time for human evaluation. However, the different ROUGE metrics give different results and it is hard to judge which is the best for automatic summaries evaluation. Also it does not include complete sentences evaluation. Therefore we suggest some work needs to be done on ROUGE in the future to make it really effective.
Liang, SF
22ac6455-24fb-40d7-b9b6-f8ae62f085fd
Liang, SF
22ac6455-24fb-40d7-b9b6-f8ae62f085fd

Liang, SF (2004) Feature Selection for Summarising: The Sunderland DUC 2004 Experience. Document Understanding Conference 2004.

Record type: Conference or Workshop Item (Paper)

Abstract

In this paper we describe our participation in task 1-very short single-document summaries in DUC 2004. The task chosen is related to our research project, which aims to produce abstracting summaries to improve search engine result summaries. DUC allowed us to produce summaries no longer than 75 characters, therefore we focused on feature selection to produce a set of key words as summaries instead of complete sentences. Three descriptions of our summarisers are given. Each of the summarisers performs very differently in the six ROUGE metrics. One of our summarisers which uses a simple algorithm to produce summaries without any supervised learning or complicated NLP technique performs surprisingly well among different ROUGE evaluations. Finally we give an analysis of ROUGE and participants’ results. ROUGE is an automatic evaluation of summaries package, which uses n-gram matching to calculate the overlapping between machine and human summaries, and indeed saves time for human evaluation. However, the different ROUGE metrics give different results and it is hard to judge which is the best for automatic summaries evaluation. Also it does not include complete sentences evaluation. Therefore we suggest some work needs to be done on ROUGE in the future to make it really effective.

Text
DUC_2004.pdf - Other
Download (104kB)

More information

Published date: 2004
Venue - Dates: Document Understanding Conference 2004, 2004-01-01
Organisations: Electronics & Computer Science

Identifiers

Local EPrints ID: 265176
URI: http://eprints.soton.ac.uk/id/eprint/265176
PURE UUID: cc9dc762-8b89-49d0-8459-fc56a0bab47c

Catalogue record

Date deposited: 14 Feb 2008 12:38
Last modified: 14 Mar 2024 08:04

Export record

Contributors

Author: SF Liang

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×