Feature Selection for Summarising: The Sunderland DUC 2004 Experience
Feature Selection for Summarising: The Sunderland DUC 2004 Experience
In this paper we describe our participation in task 1-very short single-document summaries in DUC 2004. The task chosen is related to our research project, which aims to produce abstracting summaries to improve search engine result summaries. DUC allowed us to produce summaries no longer than 75 characters, therefore we focused on feature selection to produce a set of key words as summaries instead of complete sentences. Three descriptions of our summarisers are given. Each of the summarisers performs very differently in the six ROUGE metrics. One of our summarisers which uses a simple algorithm to produce summaries without any supervised learning or complicated NLP technique performs surprisingly well among different ROUGE evaluations. Finally we give an analysis of ROUGE and participants’ results. ROUGE is an automatic evaluation of summaries package, which uses n-gram matching to calculate the overlapping between machine and human summaries, and indeed saves time for human evaluation. However, the different ROUGE metrics give different results and it is hard to judge which is the best for automatic summaries evaluation. Also it does not include complete sentences evaluation. Therefore we suggest some work needs to be done on ROUGE in the future to make it really effective.
Liang, SF
22ac6455-24fb-40d7-b9b6-f8ae62f085fd
2004
Liang, SF
22ac6455-24fb-40d7-b9b6-f8ae62f085fd
Liang, SF
(2004)
Feature Selection for Summarising: The Sunderland DUC 2004 Experience.
Document Understanding Conference 2004.
Record type:
Conference or Workshop Item
(Paper)
Abstract
In this paper we describe our participation in task 1-very short single-document summaries in DUC 2004. The task chosen is related to our research project, which aims to produce abstracting summaries to improve search engine result summaries. DUC allowed us to produce summaries no longer than 75 characters, therefore we focused on feature selection to produce a set of key words as summaries instead of complete sentences. Three descriptions of our summarisers are given. Each of the summarisers performs very differently in the six ROUGE metrics. One of our summarisers which uses a simple algorithm to produce summaries without any supervised learning or complicated NLP technique performs surprisingly well among different ROUGE evaluations. Finally we give an analysis of ROUGE and participants’ results. ROUGE is an automatic evaluation of summaries package, which uses n-gram matching to calculate the overlapping between machine and human summaries, and indeed saves time for human evaluation. However, the different ROUGE metrics give different results and it is hard to judge which is the best for automatic summaries evaluation. Also it does not include complete sentences evaluation. Therefore we suggest some work needs to be done on ROUGE in the future to make it really effective.
Text
DUC_2004.pdf
- Other
More information
Published date: 2004
Venue - Dates:
Document Understanding Conference 2004, 2004-01-01
Organisations:
Electronics & Computer Science
Identifiers
Local EPrints ID: 265176
URI: http://eprints.soton.ac.uk/id/eprint/265176
PURE UUID: cc9dc762-8b89-49d0-8459-fc56a0bab47c
Catalogue record
Date deposited: 14 Feb 2008 12:38
Last modified: 14 Mar 2024 08:04
Export record
Contributors
Author:
SF Liang
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics