Preservation Components, Workflows, Planning and Platform integration

Deprecated page This page is mostly deprecated. The most up to date information is published at http://wiki.opf-labs.org/display/SP/Cross+Project+Integration+Issues

About

The page has been started to document the results of the information architecture meeting held in Vienna May 29 - 31 May 2012. It is intended to document the methods developed for creating and registering workflow components in Scape.

The discussions at the meeting were based on the Requirements for Platform and specifically the Document GoalsandrequirementsforPTfromtheperspectiveofPW.pdf.

Introduction

A workflow component provides a workflow fragement with well defined interfaces and semantic descriptions. Workflow components must adhere to a component profile developed in this context. Workflow components can be discovered in a registry based on their annotations. Workflow components are composable and can be used to create composite workflows. Workflow components explicitly define dependencies on software packages (e.g. tools) which can be checked on the Platform.

Use Cases

UC1 publish a tool as component

1. publish toolspec 
2. write component spec (xml/rdf?) that describes what the component does using a controlled vocabulary this includes for CC+QA declaring which properties are measured
3. toolspec+componentspec generates t2flow declared as "acion/cc/qa profile", including all descriptive information (what the component does)
4. generator takes toolspec and component spec and generates t2flow workflow file
   1. for actions, this will be a complete workflow
   2. for characterisation and QA, it will be a skeleton including all required output ports. 
   3. any dependencies declared in the component spec will be declared as dependencies of that component (flattening out)
5. open t2flow in Taverna and finalise profile description / annotations
   1. for actions, this is just to validate
   2. for cc+qa, finishing the profile will often include adding nodes to parse output and map it onto the output ports corresponding to properties
6. validate and publish using the workbench

 
 #dependencies: a workflow should declare a set of dependencies. This can be generated into the t2flow annotations from the toolspec 
 # The exact identifier to use for this has to be defined carefully taking into account the package repository, package, version
 for a tool, the published workflow would use the commandline integration. what about publishing the webservice as well?
 # there should be some way of generating a webservice version and annotate it in a way that links to the tool but makes clear its a web service

UC2 publish a workflow as component
[0. create the workflow in Taverna]
1. declare a profile
2. describe what the component does
3. complete the profile and validate
 this is the same as step 5 of UC1
4. validate and publish using Taverna
## instead of using a generator to generate the t2flow, the tool PLUGIN could load the toolspec AND the component spec and generate the t2flow

UC3 query components
just use the myexperiments query api
# check expressiveness
 > there is a defined vocabulary on myexperiment that is supported by the sparql endpoint, and this is stored external to the workflow  but derived from within
 > The querying will refer to annotations in the workflow, so they will need to be extracted and exposed too

UC4 retrieve a component 
just use the myexperiment api and the Taverna workbench to download the t2flow

Meeting Outcomes

Tasks

• Specify model and all needed requirements (very detailed) (TUW) 2012-06-30
  • profiles: workflow and component categories (with input from the component packages and KEEPS)
  • interfaces for each component profile
  • attributes: attributes required to describe each profile
• Taverna profiles (MAN) 2012-07-31
• Support profiles in Taverna (MAN) 2012-07-31
  • Associate workflow to a profile
  • Annotate workflow according to profile
  • Validate workflow according to profile

• Toolspec update (KEEPS) 2012-08-31
• Component spec schema (KEEPS) 2012-08-31

SCAPE component profiles and examples (TUW) 2012-08-31

• Action migration, characterization, QA, action plans, experiment workflows 

CHECKPOINT: Component description, publication, discovery and invocation white-paper handed over to TC (TUW) 2012-09-30

• Component T2Flow generator (KEEPS) (waiting for component examples)
• Develop updated tool wrapper (only command-line) and create packages for each profile (KEEPS) (waiting for component examples)
• Generate and complete all action tool components (KEEPS) (waiting for updated toolwrapper)
• Publish all action tool components in myExperiment (KEEPS) (waiting for updated toolwrapper)
• Support annotation extraction and query in myExperiment (MAN) 2012-09-30
• Provide reference node to planning (AIT) 2012-10-31
• Extract and validate dependencies on the Platform (AIT) 2012-11-30

STATUS 2013-03-19:

• DONE: Taverna component plugin 0.9 published. (MAN)
• Component validation still to do. (MAN)
• Work ongoing on component ontologies. (KEEPS/TUW)
  • Blocks: sample components
  • Blocks: t2flow generator (sample components to be used as templates)
  • Blocks: toolwrapper
• Work ongoing on component descriptor (how to embed tool license info, which is not present in Debian package descriptor).
  • Blocks: t2flow generator
  • Blocks: toolwrapper
• Advanced component querying in myExperiment (with SPARQL) ongoing. (MAN)
  • Basic querying — always for direct properties of a component — done.
• Tool components blocked by toolwrapper

Task dependencies

http://sokrates.ifs.tuwien.ac.at:8080/rid=1KN3JX913-N4B5DN-BC/20120531-task_dependencies.cmap