# Suite fails on restart after extending length

**URL:** <https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620>\
**Category:** Unified Model\
**Tags:** ARCHER2\
**Created:** [21 January 2025 12:16 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620 "2025-01-21T12:16:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlistairDuffey](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/alistairduffey/32/294_2.png) [@AlistairDuffey](https://cms-helpdesk.ncas.ac.uk/u/AlistairDuffey)\
**Post date:** [21 January 2025 12:16 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620/1 "2025-01-21T12:16:51Z")

</div>

Hi,

I have a suite u-dm204, running UKESM1-0-LL, which worked fine through a 20-year run. I then tried to extend and restart but the suite has now failed on coupled.

Specifically, I updated the length to 35 years, and then ran ‘rose suite-run --restart’, but i find that coupled job submits okay, and then fails immediately on starting. The suite is running on archer2, my username is aduffeyum.

I can’t see anything obvious in the log files. job.err shows a sequence of errors as follows:

_MPICH ERROR [Rank 599] [job id 8540910.0] [Tue Jan 21 11:51:23 2025] [nid003425] - Abort(1) (rank 599 in comm 288): application called MPI\_Abort(comm=0xC4000002, 1) - process 599_

Any suggestions on where to start with debugging would be much appreciated!

Thanks,

Alistair

---

<div class="post-metadata">

**Author:** ![grenville](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/grenville/32/29_2.png) [@grenville](https://cms-helpdesk.ncas.ac.uk/u/grenville)\
**Post date:** [21 January 2025 16:41 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620/2 "2025-01-21T16:41:32Z")

</div>

Alistair

Look in `/home/n02/n02/aduffeyum/cylc-run/u-dm204/work/20550101T0000Z/coupled/ocean.output`

```auto
===>>> : E R R O R
         ===========

  stpctl: the zonal velocity is larger than 20 m/s

```

It might be worth trying to run with a shorter ocean time step to get past this (then revert the time step if successful.)

Grenville

---

<div class="post-metadata">

**Author:** ![AlistairDuffey](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/alistairduffey/32/294_2.png) [@AlistairDuffey](https://cms-helpdesk.ncas.ac.uk/u/AlistairDuffey)\
**Post date:** [22 January 2025 11:51 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620/3 "2025-01-22T11:51:11Z")

</div>

Hi Grenville,

Thanks very much for getting back to me on this.

I tried halving the ocean timestep (from 32 to 64 steps per day), but the suite failed again with the same error in the ocean.output file. Were you thinking a bigger change to the timestep than this?

Alternatively, is there a way for me to take the suite back a cycle to see if that helps?

---

<div class="post-metadata">

**Author:** ![grenville](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/grenville/32/29_2.png) [@grenville](https://cms-helpdesk.ncas.ac.uk/u/grenville)\
**Post date:** [23 January 2025 12:58 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620/4 "2025-01-23T12:58:49Z")

</div>

Hi Alistair

Changing the ocean time step was a very long shot. It is odd that the model fails so quickly - that might indicate something wrong with input data. I’m looking again, but you might check that all the required files are present for the restart. An option might be to start a new run (with bit compare options set) with what yuo think are the correct start files.

Grenville

---

<div class="post-metadata">

**Author:** ![system](https://europe1.discourse-cdn.com/flex013/uploads/cms_support/original/1X/1fd2411499ffcbc299fe756cd5cdf26e44956558.png) [@system](https://cms-helpdesk.ncas.ac.uk/u/system)\
**Post date:** [22 February 2025 12:59 UTC](https://cms-helpdesk.ncas.ac.uk/t/suite-fails-on-restart-after-extending-length/1620/5 "2025-02-22T12:59:15Z")

</div>

Closed on 22 February 2025 12:59 UTC

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
