Trendminer: an architecture for real time analysis of social media text
Preotiuc-Pietro, Daniel, Samangooei, Sina, Cohn, Trevor, Gibbins, Nicholas and Niranjan, Mahesan (2012) Trendminer: an architecture for real time analysis of social media text. In, 6th International AAAI Conference on Weblogs and Social Media (ICWSM-12), Dublin, IE, 05 - 07 Jun 2012. 5pp.
- Publishers print
Restricted to System admin
The emergence of online social networks (OSNs) and the accompanying availability of large amounts of data, pose a number of new natural language processing (NLP) and computational challenges. Data from OSNs is different to data from traditional sources (e.g. newswire). The texts are short, noisy and conversational. Another important issue is that data occurs in a real-time streams, needing immediate analysis that is grounded in time and context. In this paper we describe a new open-source framework for efficient text processing of streaming OSN data (available at www.trendminer-project.eu). Whilst researchers have made progress in adapting or creating text analysis tools for OSN data, a system to unify these tasks has yet to be built. Our system is focused on a real world scenario where fast processing and accuracy is paramount. We use the MapReduce framework for distributed computing and present running times for our system in order to show that scaling to online scenarios is feasible.We describe the components of the system and evaluate their accuracy. Our system supports easy integration of future modules in order to extend its functionality.
|Item Type:||Conference or Workshop Item (Paper)|
|Subjects:||H Social Sciences > HE Transportation and Communications
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
|Divisions:||Faculty of Physical Sciences and Engineering > Electronics and Computer Science > Web & Internet Science
|Date Deposited:||08 Jun 2012 08:44|
|Last Modified:||21 Aug 2012 15:54|
|Further Information:||Google Scholar|
|RDF:||RDF+N-Triples, RDF+N3, RDF+XML, Browse.|
Actions (login required)