Using Jupyter for reproducible scientific workflows
Using Jupyter for reproducible scientific workflows
Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where domain-specific software was exposed to the Jupyter environment. This enables high level control of simulations and computation, interactive exploration of computational results, batch processing on HPC resources, and reproducible workflow documentation in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress toward more reproducible and reusable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.
36-46
Beg, Marijan
5c7cc1ff-f244-471f-b964-9f24e0628153
Taka, Juliette
c4b5babd-fa2d-447a-ae86-7f40df73d5b7
Kluyver, Thomas
d3bd4245-1a99-4268-b34f-8f9f0153465e
Konovalov, Alexander
f2f374fd-5e05-4ed5-b012-80f5d37064be
Ragan-Kelley, Min
0630f6e0-9643-4da6-aabe-e56c0d1de8b0
Thiéry, Nicolas M.
1a47f2f8-7a90-49c1-b4bf-667cc5c9824c
Fangohr, Hans
9b7cfab9-d5dc-45dc-947c-2eba5c81a160
1 March 2021
Beg, Marijan
5c7cc1ff-f244-471f-b964-9f24e0628153
Taka, Juliette
c4b5babd-fa2d-447a-ae86-7f40df73d5b7
Kluyver, Thomas
d3bd4245-1a99-4268-b34f-8f9f0153465e
Konovalov, Alexander
f2f374fd-5e05-4ed5-b012-80f5d37064be
Ragan-Kelley, Min
0630f6e0-9643-4da6-aabe-e56c0d1de8b0
Thiéry, Nicolas M.
1a47f2f8-7a90-49c1-b4bf-667cc5c9824c
Fangohr, Hans
9b7cfab9-d5dc-45dc-947c-2eba5c81a160
Beg, Marijan, Taka, Juliette, Kluyver, Thomas, Konovalov, Alexander, Ragan-Kelley, Min, Thiéry, Nicolas M. and Fangohr, Hans
(2021)
Using Jupyter for reproducible scientific workflows.
Computing in Science and Engineering, 23 (2), , [9325550].
(doi:10.1109/MCSE.2021.3052101).
Abstract
Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where domain-specific software was exposed to the Jupyter environment. This enables high level control of simulations and computation, interactive exploration of computational results, batch processing on HPC resources, and reproducible workflow documentation in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress toward more reproducible and reusable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.
Text
MCSE3052101
- Accepted Manuscript
More information
Accepted/In Press date: 7 January 2021
e-pub ahead of print date: 15 January 2021
Published date: 1 March 2021
Additional Information:
Funding Information:
This work was supported in part by the Horizon 2020 European Research Projects OpenDreamKit (676541) and PaNOSC (823852), and in part by the EPSRC Programme grant on Skyrmionics under Grant EP/ N032128/1.
Publisher Copyright:
© 1999-2011 IEEE.
Identifiers
Local EPrints ID: 447441
URI: http://eprints.soton.ac.uk/id/eprint/447441
ISSN: 1521-9615
PURE UUID: 7c6fe769-2911-4523-bd61-00d5654093aa
Catalogue record
Date deposited: 11 Mar 2021 17:36
Last modified: 17 Mar 2024 03:35
Export record
Altmetrics
Contributors
Author:
Marijan Beg
Author:
Juliette Taka
Author:
Thomas Kluyver
Author:
Alexander Konovalov
Author:
Min Ragan-Kelley
Author:
Nicolas M. Thiéry
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics