Is complex query answering really complex?
Is complex query answering really complex?
Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be as complex as we think, as the way they are built distorts our perception of progress in this field. For example, we find that in these benchmarks most queries (up to 98% for some query types) can be reduced to simpler problems, e.g., link prediction, where only one link needs to be predicted. The performance of state-of-the-art CQA models decreses significantly when such models are evaluated on queries that cannot be reduced to easier types. Thus, we propose a set of more challenging benchmarks composed of queries that require models to reason over multiple hops and better reflect the construction of real-world KGs. In a systematic empirical investigation, the new benchmarks show that current methods leave much to be desired from current CQA methods.
Gregucci, Cosimo
d0dfa9ed-ddb4-4380-9d8f-e9a362e99db7
Xiong, Bo
d8c3ce0a-07ac-43f8-bd67-f230c6cbc1ec
Hernandez, Daniel
39723173-ccff-4015-b506-cec5aec76936
Loconte, Lorenzo
3b65c31f-30cd-417e-a332-f7683a35405a
Minervini, Pasquale
a6cbde0c-f142-4c28-bd24-bc3887c90400
Staab, Steffen
bf48d51b-bd11-4d58-8e1c-4e6e03b30c49
Vergari, Antonio
dc94a546-2cf8-466e-a25c-d30337b40324
Gregucci, Cosimo
d0dfa9ed-ddb4-4380-9d8f-e9a362e99db7
Xiong, Bo
d8c3ce0a-07ac-43f8-bd67-f230c6cbc1ec
Hernandez, Daniel
39723173-ccff-4015-b506-cec5aec76936
Loconte, Lorenzo
3b65c31f-30cd-417e-a332-f7683a35405a
Minervini, Pasquale
a6cbde0c-f142-4c28-bd24-bc3887c90400
Staab, Steffen
bf48d51b-bd11-4d58-8e1c-4e6e03b30c49
Vergari, Antonio
dc94a546-2cf8-466e-a25c-d30337b40324
Gregucci, Cosimo, Xiong, Bo, Hernandez, Daniel, Loconte, Lorenzo, Minervini, Pasquale, Staab, Steffen and Vergari, Antonio
(2025)
Is complex query answering really complex?
International Conference on Machine Learning 2025, Vancouver, Canada, Vancouver, Canada.
11 - 19 Jul 2025.
30 pp
.
(In Press)
Record type:
Conference or Workshop Item
(Paper)
Abstract
Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be as complex as we think, as the way they are built distorts our perception of progress in this field. For example, we find that in these benchmarks most queries (up to 98% for some query types) can be reduced to simpler problems, e.g., link prediction, where only one link needs to be predicted. The performance of state-of-the-art CQA models decreses significantly when such models are evaluated on queries that cannot be reduced to easier types. Thus, we propose a set of more challenging benchmarks composed of queries that require models to reason over multiple hops and better reflect the construction of real-world KGs. In a systematic empirical investigation, the new benchmarks show that current methods leave much to be desired from current CQA methods.
Text
10624_Is_Complex_Query_Answeri
- Accepted Manuscript
More information
Accepted/In Press date: 27 May 2025
Venue - Dates:
International Conference on Machine Learning 2025, Vancouver, Canada, Vancouver, Canada, 2025-07-11 - 2025-07-19
Identifiers
Local EPrints ID: 502877
URI: http://eprints.soton.ac.uk/id/eprint/502877
PURE UUID: 0d3fcbb5-b14a-4d56-b5c9-b6bfe34854dc
Catalogue record
Date deposited: 10 Jul 2025 17:20
Last modified: 22 Aug 2025 02:13
Export record
Contributors
Author:
Cosimo Gregucci
Author:
Bo Xiong
Author:
Daniel Hernandez
Author:
Lorenzo Loconte
Author:
Pasquale Minervini
Author:
Steffen Staab
Author:
Antonio Vergari
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics