State of CWL support?

[fer-rum]

Hi,

What is the current state of supporting the common workflow language (CWL) in AIIDA? I found hints that someone built a plugin for that but sadly no link to code was provided.

Is anything in this direction planned, or is there another workflow description that is supported?

Thanks in advance

[mbercx]

Hi @fer-rum. 👋 I vaguely remember this being mentioned in the meeting whose notes you linked to, but I'm afraid I'm not very familiar with the details. I'm assuming the plugin mentioned there uses/is integrated into the aiida-shell plugin:

https://github.com/sphuber/aiida-shell

I'll ping @sphuber here, who developed the aiida-shell plugin and will surely know more.

[sphuber]

The aiida-shell plugin does not directly provide CWL integration with AiiDA. However, it is the first step towards this. Way back when I started developing it, I actually experimented with implementing a package that would turn AiiDA into a CWL engine. This was only for a subset of the CWL syntax, but in principle it worked.

I wrote pycwl to parse a CWL document into Python objects, which I then used to run it using aiida-shell. I never continued the work since there was no concrete use-case and I wasn't using CWL myself. But I could dig up the code and make it available if you are interested. It would probably require some work to update it since aiida-shell has changed and improved a lot, so the old code is probably no longer compatible.

It would also help if you could share an example of a CWL workflow that you would like to run, if you are able to share that.

[rikigigi]

I'm also interested in a cwl integration. If there is any ongoing project I'd like to join! :-)

[sphuber]

I have a repo (not public as of yet) with a proof of concept, but that is based off of the very first version of aiida-shell that was never released. I would have to rewrite it to be based on the current model which is doable, but would take some work. It would be great if you could share some (abstract) example of what you would like to run to see if that subset of CWL is even implementable with AiiDA. When I find some time, I will try to migrate the concept to the modern aiida-shell and then make it public so we can iterate there on how to further develop it.

[sphuber]

@fer-rum @rikigigi I have updated my proof-of-principle to be compatible with the latest aiida-shell and made it publicly available: https://github.com/sphuber/aiida-cwl
I suggest that if you are interested to continue to the discussion there. It would be great if you could give it a try and suggest in an issue what CWL functionality is missing for your use case. Note that the amount of CWL schema that is implemented currently is still quite limited.