A RAG-based question-answering solution for cyber-attack investigation and attribution

Abstract

In the constantly evolving field of cybersecurity, it is imper- ative for analysts to stay abreast of the latest attack trends and perti- nent information that aids in the investigation and attribution of cyber- attacks. In this work, we introduce the first question-answering (QA) model and its application that provides information to the cybersecu- rity experts about cyber-attacks investigations and attribution. Our QA model is based on Retrieval Augmented Generation (RAG) techniques together with a Large Language Model (LLM) and provides answers to the users’ queries based on either our knowledge base (KB) that contains curated information about cyber-attacks investigations and attribution or on outside resources provided by the users. We have tested and evalu- ated our QA model with various types of questions, including KB-based, metadata-based, specific documents from the KB, and external sources- based questions. We compared the answers for KB-based questions with those from OpenAI’s GPT-3.5 and the latest GPT-4o LLMs. Our pro- posed QA model outperforms OpenAI’s GPT models by providing the source of the answers and overcoming the hallucination limitations of the GPT models, which is critical for cyber-attack investigation and attribu- tion. Additionally, our analysis showed that when the RAG QA model is given few-shot examples rather than zero-shot instructions, it gener- ates better answers compared to cases where no examples are supplied in addition to the query.

More information

Identifiers

ORCID for Erisa Karafili: orcid.org/0000-0002-8250-4389

Catalogue record

Export record