Cylc install hangs

JULES
1

Hello

I have been away for the last year and a rose suite (u-dg545) that I had running fine before I left is now failing. It was running on the new cylc2 with lotus2. At the moment, when I try “cylc install” it just hangs. Nothing is written to my local “cylc-run” folder, so I’m not sure where to look for error messages. Any tips for how I could start to try to solve this would be very much appreciated.

Many thanks

Sarah

2

Hi Sarah.

It might be related to this issue :right_arrow: Restarting after puma2 reboot.

Let us know how you get on.

Cheers

Jonny

3

Hi Sarah

JASMIN had some power issues last week and is still only partially back. See Status - Centre for Environmental Data Analysis. One place where cylc install could be stalling is where it creates the share and work directories on the scratch file system.

David

4

Hello I think I have the same issue. When I run cylc vip on my suite it just hangs at the cylc install stage. This is all I see when I run cylc vip:

(base) \[emg97@cylc2 u-cy382\]$ cylc vip
$ cylc validate /home/users/emg97/roses/u-cy382
WARNING - Backward compatibility mode ON - support for suite.rc files will be removed at 8.7.0
Valid for cylc-8.6.4
$ cylc install /home/users/emg97/roses/u-cy382

But my ssh keys are all as they should be an I am able to access everything else on cylc2@jasmin like my roses directory and the cylc TUI.

5

Hi Sarah and Elliot

I’ve done some experiments trying to install Sarah’s suite (u-dg545). I too get cylc install hanging with my normal Cylc settings. The problem appears to be due to /work/scratch-pw4 currently being inaccessible. When Cylc installs a suite under $HOME/cylc-run/<workflow-id>, the share and work subdirectories are usually created as symbolic links to directories of the same name under /work/scratch-pw4/$USER/cylc-run/<workflow-id>. This behaviour is set by the following lines in /apps/jasmin/metomi/metomi-site/etc/cylc/flow/8/global.cylc:

[install]
    [[symlink dirs]]
        [[[localhost]]]
            share = /work/scratch-pw4/$USER
            work = /work/scratch-pw4/$USER

For more details see Platform Configuration — Cylc 8.6.3 documentation and Global Configuration — Cylc 8.6.3 documentation.

You can either wait for /work/scratch-pw4 to become available again, or you can change the global.cylc settings so that the share and work directories are elsewhere. How to change these settings is explained at the start of the second documentation file linked to above. I was able to install u-dg545 after creating a file ~/.cylc/flow/8/global.cylc, copying the above five lines beginning [install] to it, and changing /work/scratch-pw4/$USER to ~/tmp. I haven’t tried running the suite. If it creates large files, I might run out of space with the share and work directories under my home directory like this.

Good luck
David

6

Thank you so much, David. That makes complete sense. I didn’t realise the power outage was still an ongoing issue. I just assumed being away for so long that something had changed.