ARCHER2 cylc version suddenly reverted?

Rose/Cylc and FCM
,
1

Hi,

I previously had an issue where tasks were failing to submit from puma2 to ARCHER2. Ros fixed this by noticing my ARCHER2 ~/.bash_profile was pointing to a very old UM environment with cylc version 7: https://cms-helpdesk.ncas.ac.uk/t/submit-failed-puma2-to-archer2/2121/3?u=racheldiamond

My suite u-ea675 was running fine but this morning around 6am stopped with similar submit-failed errors. I checked on ARCHER2 and again have cylc --version 7.8.12 (instead of cylc 8). Nothing in my bash profile has changed.

How should I fix this? I’m a bit nervous to try lots of different restart methods as u-ea675 is meant to be bit-comparable to another running suite.

Best wishes,

Rachel

2

Hi Rachel.

Couple of questions…

  1. What is your PUMA2/ARCHER2 username?
  2. What do you get when you run cylc --version on PUMA2?
  3. What do you get when you run which cylc on PUMA2

This is what I get…

$ cylc --version

8.6.3

$ which cylc

/home/n02/n02/fcm/metomi/bin/cylc

It’s possible too that even though you’ve made sure you’re using Cylc 8 on PUMA2 that it is being set to use version 7 on ARCHER2. I have export CYLC_VERSION=8 in my startup scripts on PUMA2 and ARCHER2.

Also, if you’re restarting from a previously stopped job which ran with Cylc 7 then it will automatically pick version 7 again even if you have version 8 hardcoded in your startup scripts. Restarting using a different Cylc version is possible but is fraught with potential potholes.

Cheers

Jonny

3

Hi,

My PUMA2 and ARCHER2 username is radiam24 (met office username is racheldiamond). On PUMA2 I get cylc version 8.6.3, and which cylc >
/home/n02/n02/fcm/metomi/bin/cylc.

I’m trying to restart a job that was running fine yesterday and I haven’t changed anything since then, just started suddenly getting submit-failed messages this morning. So it’s possible it’s not related to the CYLC version at all, I just assumed it was because of Ros’s fix on the previous problem in July.

Thanks!

Rachel

4

Thanks for that info Rachel. It’s possible that the project you’re submitting under has reached budget. We’ll get back to you on this.

$ cat /home/n02/n02/radiam24/roses/u-ea675/rose-suite.conf|grep -i archer2_group

ARCHER2_GROUP='n02-SURFEIT'

Cheers

Jonny