Benchmarking Workflow Discovery: A Case Study From Bioinformatics

Download

PDF Download (423Kb)

Description/Abstract

Automation in science is increasingly marked by the use of workflow technology. The sharing of workflows through repositories supports the verifability, reproducibility and extensibility of computational experiments. However, the subsequent discovery of workflows remains a challenge, both from a sociological and technological viewpoint. Based on a survey with participants from 19 laboratories, we investigate current practices in workflow sharing, re-use and discovery amongst life scientists chiefly using the Taverna workflow management system. To address their perceived lack of effective workflow discovery tools, we go on to develop benchmarks for the evaluation of discovery tools, drawing on a series of practical exercises. We demonstrate the value of the benchmarks on two tools: one using graph matching, the other relying on text clustering.