# Suite says that it failed but it's still running

**URL:** https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322
**Category:** Rose/Cylc and FCM
**Tags:** JASMIN, JULES
**Created:** [29 January 2024 10:03 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322 "2024-01-29T10:03:38Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)
#### Post date: [29 January 2024 10:03 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322/1 "2024-01-29T10:03:39Z")

</div>

My u-da046 JULES suite is running on JASMIN. The cylc GUI said that it failed in the cycle for years 1860-1870. But there isn’t anything in the job.err or job.out or job-activity.log files that says that it failed. I tried to poll the suite, but it still says that it failed, even though it is still running as job ID 17377026 on par-single on SLURM. I reset the state of the job as running. Was that the right thing to do? Is there any explanation for it saying that it failed in the cylc GUI?  
Patrick

---

<div class="post-metadata">

### Author: ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)
#### Post date: [29 January 2024 11:45 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322/2 "2024-01-29T11:45:07Z")

</div>

Hi Patrick,

These symptoms usually indicate an intermittent issue with the slurm command. Cylc uses `squeue` to query the scheduler to see if the job is still running. If the command doesn’t work and doesn’t error, cylc will think the task has exited the queue and thus flags it as failed. It’s a known issue.

The job.status file has the tell-tale lines:

```auto
CYLC_BATCH_SYS_EXIT_POLLED=2024-01-29T08:07:18Z
CYLC_JOB_EXIT=SUCCEEDED
CYLC_JOB_EXIT_TIME=2024-01-29T10:41:10Z

```

Cheers,  
Ros.

---

<div class="post-metadata">

### Author: ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)
#### Post date: [29 January 2024 12:28 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322/3 "2024-01-29T12:28:42Z")

</div>

Hi Ros  
Thanks for explaining!  
Much appreciated.  
Patrick

---

<div class="post-metadata">

### Author: ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)
#### Post date: [31 January 2024 12:29 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-says-that-it-failed-but-its-still-running/1322/4 "2024-01-31T12:29:28Z")

</div>

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.