UKCA submit-failed

1

Hi there,
I copied a suite (u-dr256) from a working suite u-dq164. However, it did not run with rose errors in “install_ancil” and with state of “submit-failed”. The further error messages in Rose report are not useful, as they all show: ERROR: file not found: /home/n02/n02/emxin/cylc-run/u-dr256/log/job/20200201T0000Z/install_ancil/01/job.out.

Then I went to cycle-run log at: /home/n02/n02/emxin/cylc-run/u-dr256/log/job/20200201T0000Z/install_cold/01
the job.out file shows below message:

  1. epcc-setup-env
  2. load-epcc-module
  3. cce/15.0.0
  4. cray-hdf5-parallel/1.12.2.1
  5. cray-netcdf-hdf5parallel/4.9.0.1
  6. um/2023.06

2025-07-03T13:08:29Z INFO - started
[INFO] Configuration: /work/n02/n02/emxin/cylc-run/u-dr256/app/install_cold/
[INFO] file: rose-app.conf
[INFO] optional key: (ukesm_amip)
[INFO] optional key: (n96e)
[INFO] optional key: (gregorian)
[INFO] optional key: (archer2)
[INFO] export ANCIL_ATM_RES=n96e
[INFO] export ANCIL_OCN_RES=_orca1
[INFO] export ANCIL_REV=''
[INFO] export ANCIL_ROOT=/work/y07/shared/umshared/ancil/data/ancil_versions
[INFO] export ANCIL_VN=GA7.1/v2
[INFO] export PATH=/mnt/lustre/a2fs-work1/work/y07/shared/umshared/metomi/rose-2019.01.3/bin:/opt/cray/pe/netcdf-hdf5parallel/4.9.0.1/bin:/opt/cray/pe/hdf5-parallel/1.12.2.1/bin:/opt/cray/pe/hdf5/1.12.2.1/bin:/opt/cray/pe/cce/15.0.0/binutils/x86_64/x86_64-pc-linux-gnu/bin:/opt/cray/pe/cce/15.0.0/binutils/cross/x86_64-aarch64/aarch64-linux-gnu/../bin:/opt/cray/pe/cce/15.0.0/utils/x86_64/bin:/opt/cray/pe/cce/15.0.0/bin:/opt/cray/pe/cce/15.0.0/cce-clang/x86_64/bin:/work/n02/n02/emxin/cylc-run/u-dr256/bin:/work/y07/shared/umshared/metomi/bin:/home/n02/n02/emxin/miniconda3/envs/py12/bin:/home/n02/n02/emxin/miniconda3/condabin:/work/y07/shared/utils/core/bolt/0.8/bin:/mnt/lustre/a2fs-work4/work/y07/shared/utils/core/bin:/opt/cray/pe/mpich/8.1.23/ofi/crayclang/10.0/bin:/opt/cray/pe/mpich/8.1.23/bin:/opt/cray/pe/craype/2.7.19/bin:/opt/cray/pe/cce/15.0.0/binutils/cross/x86_64-aarch64/bin:/opt/cray/pe/perftools/22.12.0/bin:/opt/cray/pe/papi/6.0.0.17/bin:/opt/cray/libfabric/1.12.1.2.2.0.0/bin:/home/n02/n02/emxin/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/opt/cray/pe/bin:/work/y07/shared/umshared/bin:/work/y07/shared/utils/core/python/miniconda2/bin
[INFO] create: /work/n02/n02/emxin/cylc-run/u-dr256/share/data/etc
[INFO] symlink: /work/n02/n02/emxin/archive/u-do265/do265a.da20200201_00 <= /work/n02/n02/emxin/cylc-run/u-dr256/share/data/dr256a.ainitial
[INFO] source: /work/y07/shared/umshared/ancil/data/ancil_versions/n96e_orca1/GA7.1/v2/ancils
[INFO] install: /work/n02/n02/emxin/cylc-run/u-dr256/share/data/etc/um_ancils_gl
[INFO] source: /work/y07/shared/umshared/ancil/data/ancil_versions/n96e_orca1/GA7.1/v2/ancils
[INFO] command: true
2025-07-03T13:08:33Z INFO - succeeded

Noe that the old working suite u-dq164 also stopped working with same message, which is strange. Could you tell me what went wrong? Thanks

Best,
Xin

2

Hi Xin,

In case of ‘submit-failed’ tasks, the error message is usually in a job-activity.log file under log/job/(cycle)/(task)/NN/.
~emxin/cylc-run/u-dr256/log/job/20200201T0000Z/install_ancil//NN/job-activity.log shows failure for ‘rose host-select archer2’ command.

This could be temporary and a retry should work, otherwise try executing this command on PUMA. It should return one of the ‘ln0*’ nodes.

–
Mohit

3

Yes, it works now. Thanks.
Xin