Output details
36 - Communication, Cultural and Media Studies, Library and Information Management
University of Liverpool
PYRODS (Python Language Workflow Orchestration System)
The PYRODS workflow orchestration system was researched, developed, and published as an independent software library at the University of Liverpool from the period 2010-2013. The research was funded by the EU and the JISC to develop a virtual workflow environment that supports the analysis, synthesis, and access of scientific data, managed in an archive. PYRODS allows researchers to design and execute flexible processing sequences that represent complex data analysis. These steps can be archived for later use (“reproducible science”), either to verify experimental results or to create new data products. PYRODS also provides a GUI to allow any level of user from a variety of disciplines to design these workflows in a drag-and-drop manner.
The research innovation consists of a model to implement preservation processes as virtualised workflows, with the data and workflow steps stored in archived collections for reuse or verification. The implementation required optimisation of management processes across different levels of abstraction and a modular structuring of workflow execution.
PYRODS is designed to meet the following needs for reproducible science :
• The software design and structure arises from this need for an environment that supports the sharing, adaptation and automated re-execution of workflows for the conditioning, reduction and analysis of research data.
• Interoperable workflows that support re-usable and adaptive procedures for data management in scientific collaborations;
• The need to integrate multiple executable scientific workflows and storage workflows, thus removing infrastructure dependencies;
• The need to be able to share and re-execute scientific workflows with minimal intervention;
• The need to be able to save the workflows as digital objects in the data grid to be read or modified in the future.
PYRODS is used internationally for executing processing workflows as part of archival systems in many services, including Sanger Institute, IN2P3, and NASA.
