Hi
Thanks for the guidance to get me moving with cylc7.
Cylc8 has some enhancements that we hope to use, so I’ve had initial go with that, but fail to establish the link to slurm, yet.
May I ask
a) do you have any plans to install cylc8
b) Is there a error in this that you might spot…I had hoped $HOME/.cylc/flow/global.rc would be ok with (based on one we had on our university cluster):
[platforms]
[[archer2]]
hosts = login-4c.archer2.ac.uk
job runner = slurm
install target = localhost
(With $HOME set to a workspace directory, and cylc set up, so the “cylc install” and validate on a test case are ok; the “cylc play” achieves little and doesn’t submit to slurm)
The test case is:
cat flow.cylc
#title = “Test suite: one off local tasks”
#description = “A suite containing one to task to be run on compute node”
[scheduling]
[[graph]]
R1 = “”“onef”""
[runtime]
[[onef]]
script = “sleep 30s; echo ‘Running task oneoff’; true”
platform = archer2
[[[directives]]]
–job-name = run_oneoff
–account = n02-optclim
–time = 00:10:00
–partition = standard
–qos = short
–reservation = shortqos
–nodes = 1
–tasks-per-node = 1
Regards
Mike
Hi Mike,
Our current plans are to install cylc-8 when our replacement PUMA service comes online hopefully early next year. I haven’t played with cylc-8 since one of the very very early releases and was currently waiting for the first release/release candidate which should be soon, before taking another look.
I can’t see anything obvious wrong in the config that you posted. You’re trying to submit from edinburgh cluster is that right? To be honest I haven’t totally digested how all the communications now work between the local and remote hosts.
Out of interest what cylc enhancements are you wishing to use?
Sorry not to be much help here. If I think of anything I’ll let you know.
Regards,
Ros.
Hi Ros
Thanks for your response – I know I am on the edge of what is fair to ask, and appreciate the thoughts.
At present for initial use we are trying to use Archer2 frontend to run cylc, but are pondering using our own server in Edinburgh (or because we have to interact with rose/cylc for the UKESM, maybe PUMA). Using the frontend is as we discussed with cylc7, but maybe its not a good plan from midterm (in terms of polite use of Archer2) and we need to do otherwise. Guidance welcome.
The enhancements with cylc8 are to do with “or” constructs in the workflow – arising in the way that one task completes can determine the next task in the workflow to run
Regards
Mike
Hi Mike,
Using your own server in Edinburgh to submit Cylc suites from will almost certainly be unviable. Cylc requires non-interactive access to the remote host which is not possible with ARCHER2 being 2 factor-authentication. We are able to do this from PUMA at the moment, as we currently have a special temporary arrangement/system in place that was agreed with EPCC. Once the full ARCHER2 system comes on line PUMA will be moved into the ARCHER2 domain where communication between the 2 systems will be possible. I would recommend that in the long term you look to use PUMA.
I’d forgotten that you were currently running cylc from the ARCHER2 login nodes, in which case, I wonder if the [[archer2]] section should not have the line hosts = login-4c.archer2.ac.uk rather hosts = localhost otherwise, assuming it works in the same way as cylc7, that will cause cylc to ssh from login-4c to itself which will potentially cause problems (we see issues on Monsoon2 with cylc7 when people do this). Don’t know if that makes any difference here…
Regards,
Ros.