Hi,
I am getting a failure from a small serial job – asks for 15 minutes in the serial q.
Failure messages are:
[jobs-submit cmd] ssh -oBatchMode=yes -oConnectTimeout=8 -oStrictHostKeyChecking=no ln04 env CYLC_VERSION=8.6.0 CYLC_ENV_NAME=cylc-8.6.0-1 bash --login -c ''"'"'exec "$0" "$@"'"'"'' cylc jobs-submit --debug --utc-mode --remote-mode --clean-env --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/opt_dfols4/d400g/log/job' 20111201T0000Z/optclim_post/02
[jobs-submit ret_code] 1
[jobs-submit out] 2025-11-01T11:45:54Z|20111201T0000Z/optclim_post/02|1|None
2025-11-01T11:45:54Z [STDERR] sbatch: error: AssocMaxCpuMinutesPerJobLimit
2025-11-01T11:45:54Z [STDERR] sbatch: error: Batch job submission failed: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)
The first attempt to run this job ran out of time…. So when I triggered it again I got the above failure which I don’t think I’ve had before.
Has n02-TERRAFIRMA run out of resources? I had a very similar job run at 01:20 which took 5 mins.
Running the command interactively takes a few seconds. The job it releases (don’t ask…) is sitting in state AssocMaxCpuMinutesPerJobLimit.
Looking at Job violates accounting/QOS policy suggests, to me, that n02-TERRAFIRMA needs some more CU’s. From my POV it probably needs quite a lot….
See /home/n02/n02/tetts/cylc-run/opt_dfols4/d400g/log/job/20111201T0000Z/optclim_post/03/job-activity.log for log.
Simon