# Vn13.3 AMIP suite on Archer2 **URL:** \ **Category:** Uncategorized\ **Tags:** Monsoon2, ARCHER2\ **Created:** [16 October 2024 15:10 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539 "2024-10-16T15:10:14Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [25 October 2024 14:42 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/21 "2024-10-25T14:42:03Z")
Thanks, Ros, I’ve applied for access to hpxfer with that IP address. James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [28 October 2024 09:04 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/22 "2024-10-28T09:04:20Z")
Hi Ros, I was granted access to the hpxfer server on Friday evening. From the jasmin login node, I can run the command “ssh -Y [hpxfer2.jasmin.ac.uk](http://hpxfer2.jasmin.ac.uk)” successfully but “ssh -Y [hpxfer1.jasmin.ac.uk](http://hpxfer1.jasmin.ac.uk)” does not work. However, if I run “ssh -Y [hpxfer2.jasmin.ac.uk](http://hpxfer2.jasmin.ac.uk)” on Archer2, I get the following error. [jweber@hpxfer2.jasmin.ac.uk](mailto:jweber@hpxfer2.jasmin.ac.uk): Permission denied (publickey,gssapi-keyex,gssapi-with-mic) I set off a rerun of u-dk384 last night but received the error below on pptransfer. Lmod is automatically replacing “cce/15.0.0” with “gcc/11.2.0”. Due to MODULEPATH changes, the following have been reloaded: 1. cray-mpich/8.1.23 [WARN] file:atmospp.nl: skip missing optional source: namelist:moose\_arch [WARN] file:atmospp.nl: skip missing optional source: namelist:script\_arch [WARN] [SUBPROCESS]: Command: globus-url-copy -vb -cd -r -cc 4 -sync -cred /work/n02/n02/jweber/cred.jasmin /work/n02/n02/jweber/cylc-run/u-dk384/share/cycle/19800101T0000Z/u-dk384/19800101T0000Z/ gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk384/19800101T0000Z/ [SUBPROCESS]: Error = 1: error: Unable to check destination url for sync: gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk384/19800101T0000Z/ globus\_ftp\_client: the server responded with an error 530 530-Login incorrect. : globus\_gss\_assist: Gridmap lookup failure: Could not map /DC=uk/DC=ac/DC=jasmin/O=STFC RAL/CN=jmw240 530- 530 End. [WARN] Transfer command failed: globus-url-copy -vb -cd -r -cc 4 -sync -cred /work/n02/n02/jweber/cred.jasmin /work/n02/n02/jweber/cylc-run/u-dk384/share/cycle/19800101T0000Z/u-dk384/19800101T0000Z/ gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk384/19800101T0000Z/ [ERROR] transfer.py: Unknown Error - Return Code=1 [FAIL] Command Terminated [FAIL] Terminating PostProc… [FAIL] transfer.py \<\<‘ **STDIN** ’ [FAIL] [FAIL] ‘ **STDIN** ’ # return-code=1 2024-10-28T05:08:16Z CRITICAL - failed/EXIT Is there something else I need to do to set up the link between Archer2 and Jasmin’s hpxfer? Thanks, James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [28 October 2024 12:59 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/23 "2024-10-28T12:59:54Z")
I added a "/’ to the end of the Jasmin path and the archive appears to have worked. Not quite sure why but looks like everything is sorted, thanks for your help. James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [4 November 2024 09:36 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/24 "2024-11-04T09:36:47Z")
Hi, Just a quick follow up question. I’ve had a couple of pptransfer failures with the last one returning the error message below. [WARN] file:atmospp.nl: skip missing optional source: namelist:moose\_arch [WARN] file:atmospp.nl: skip missing optional source: namelist:script\_arch Traceback (most recent call last): File “/mnt/lustre/a2fs-work1/work/y07/shared/umshared/metomi/cylc-7.8.12/bin/cylc-message”, line 140, in main() File “/mnt/lustre/a2fs-work1/work/y07/shared/umshared/metomi/cylc-7.8.12/bin/cylc-message”, line 136, in main return record\_messages(suite, task\_job, messages) File “/mnt/lustre/a2fs-work1/work/y07/shared/umshared/metomi/cylc-7.8.12/lib/cylc/task\_message.py”, line 72, in record\_messages handle.write(‘%s %s - %s\n’ % (event\_time, severity, message)) IOError: [Errno 122] Disk quota exceeded I can’t work out if this is a disk quota error on Jasmin or Archer2. Could you advise? Thanks, James ---
**Author:** ![grenville](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/grenville/32/29_2.png) [@grenville](https://cms-helpdesk.ncas.ac.uk/u/grenville)\ **Post date:** [4 November 2024 09:59 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/25 "2024-11-04T09:59:45Z")
James The n02 /work space filled up on ARCHER. Pl try again now. Grenville ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [4 November 2024 10:01 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/26 "2024-11-04T10:01:16Z")
Thanks, Grenville. It seems to be working now. James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [8 November 2024 11:01 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/27 "2024-11-08T11:01:41Z")
Hi, u-dk655, a coy of u-dk384, has failed on atmos\_main with the below error which I haven’t seen before: MPICH ERROR [Rank 432] [job id 7996193.0] [Thu Nov 7 22:00:24 2024] [nid006161] - Abort(405395855) (rank 432 in comm 352): Fatal error in PMPI\_Send: Other MPI error, error stack: PMPI\_Send(163)…: MPI\_Send(buf=0xdea75c8, count=86, dtype=USER, dest=479, tag=8, comm=0xc400000b) failed PMPI\_Send(143)…: MPIR\_Wait\_impl(41)…: MPID\_Progress\_wait(193)…: MPIDI\_Progress\_test(89)…: MPIDI\_OFI\_handle\_cq\_error(1062): OFI poll failed (ofi\_events.c:1064:MPIDI\_OFI\_handle\_cq\_error:Input/output error - transport retry counter exceeded) srun: error: nid006132: tasks 1-55: Exited with exit code 143 srun: launch/slurm: \_step\_signal: Terminating StepId=7996193.0 slurmstepd: error: \*\*\* STEP 7996193.0 ON nid006132 CANCELLED AT 2024-11-07T22:00:25 \*\*\* srun: error: nid006132: tasks 56-62: Exited with exit code 143 srun: error: nid006134: tasks 63-76: Exited with exit code 143 srun: error: nid006134: tasks 77-125: Terminated srun: error: nid006163: tasks 441-503: Terminated srun: error: nid006156: tasks 189-251: Terminated srun: error: nid006160: tasks 315-377: Terminated srun: error: nid006161: tasks 378-440: Terminated srun: error: nid006159: tasks 252-314: Terminated srun: error: nid006151: tasks 126-188: Terminated srun: error: nid006132: task 0: Terminated srun: Force Terminated StepId=7996193.0 [FAIL] um-atmos \<\<‘ **STDIN** ’ [FAIL] [FAIL] ‘ **STDIN** ’ # return-code=143 2024-11-07T22:00:44Z CRITICAL - failed/EXIT Given problems with Archer2 storage, is this a problem with my suite or a wider issue? Thanks, James ---
**Author:** ![grenville](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/grenville/32/29_2.png) [@grenville](https://cms-helpdesk.ncas.ac.uk/u/grenville)\ **Post date:** [8 November 2024 11:19 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/28 "2024-11-08T11:19:45Z")
James We see these errors from time to time - a retrigger of the failed task should be enough. Grenville ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [8 November 2024 13:22 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/29 "2024-11-08T13:22:59Z")
Thanks, Grenville, atmos\_main completed after retriggering. James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [13 November 2024 09:44 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/30 "2024-11-13T09:44:25Z")
Hi, separate issue but it looks like the bottleneck for this run is the postproc task rather than atmos\_main. I’ve cut down the output diagnostics and increased the memory for the POSTPROC\_RESOURCE but is there anything else I can do to speed it up? Thanks, James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [18 November 2024 14:45 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/31 "2024-11-18T14:45:44Z")
Hi, pptransfer has failed with the below error message even after several retrigger attempts [WARN] file:atmospp.nl: skip missing optional source: namelist:moose\_arch [WARN] file:atmospp.nl: skip missing optional source: namelist:script\_arch [WARN] [SUBPROCESS]: Command: globus-url-copy -vb -cd -r -cc 4 -sync -cred /work/n02/n02/jweber/cred.jasmin /work/n02/n02/jweber/cylc-run/u-dk655/share/cycle/19871001T0000Z/u-dk655/19871001T0000Z/ gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk655/19871001T0000Z/ [SUBPROCESS]: Error = 1: error: Unable to list destination directory for sync: gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk655/19871001T0000Z/ globus\_ftp\_client: the server responded with an error 500 500-Command failed. : an authentication operation failed 500-globus\_gsi\_callback\_module: Could not verify credential 500-globus\_gsi\_callback\_module: Error with signing policy 500-globus\_gsi\_callback\_module: Error in OLD GAA code: The subject of the certificate “/DC=uk/DC=ac/DC=jasmin/O=STFC RAL/CN=JASMIN” does not match the signing policies defined in /home/users/jmw240/.globus/certificates/7ed47087.signing\_policy 500 End. [WARN] Transfer command failed: globus-url-copy -vb -cd -r -cc 4 -sync -cred /work/n02/n02/jweber/cred.jasmin /work/n02/n02/jweber/cylc-run/u-dk655/share/cycle/19871001T0000Z/u-dk655/19871001T0000Z/ gsiftp://gridftp1.jasmin.ac.uk/gws/nopw/j04/sheffield/jweber/archive/u-dk655/19871001T0000Z/ [ERROR] transfer.py: Unknown Error - Return Code=1 [FAIL] Command Terminated [FAIL] Terminating PostProc… [FAIL] transfer.py \<\<‘ **STDIN** ’ [FAIL] [FAIL] ‘ **STDIN** ’ # return-code=1 2024-11-18T14:32:09Z CRITICAL - failed/EXIT My credential was still in date but I have renewed it so it runs to 18th Dec anyway. Jasmin has been having issues but I think they are now resolved. Does this look like a Jasmin or Archer2 issue? Cheers, James ---
**Author:** ![mdalvi](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@mdalvi](https://cms-helpdesk.ncas.ac.uk/u/mdalvi)\ **Post date:** [18 November 2024 16:46 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/32 "2024-11-18T16:46:12Z")
Hi James, There have been some storage issues at CEDA since the 14th, especially affecting /gws/nopw/ . They seem to have fixed this today but have advised that this is ‘at risk’ [Centre for Environmental Data Analysis - Status](https://www.ceda.ac.uk/status/) ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [19 November 2024 09:11 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/33 "2024-11-19T09:11:52Z")
Thanks, Mohit. I’ll wait until Jasmin is back to normal operation and try again. James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [20 November 2024 09:49 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/34 "2024-11-20T09:49:40Z")
Hi, Jasmin seems to be working now but I’m getting a different error with pptransfer: Lmod is automatically replacing “cce/15.0.0” with “gcc/11.2.0”. Due to MODULEPATH changes, the following have been reloaded: 1. cray-mpich/8.1.23 [WARN] file:atmospp.nl: skip missing optional source: namelist:moose\_arch [WARN] file:atmospp.nl: skip missing optional source: namelist:script\_arch Traceback (most recent call last): File “/work/n02/n02/jweber/cylc-run/u-dk655/share/fcm\_make\_pp/build/bin/transfer.py”, line 451, in main() File “/work/n02/n02/jweber/cylc-run/u-dk655/share/fcm\_make\_pp/build/bin/transfer.py”, line 429, in main transfer = Transfer() File “/work/n02/n02/jweber/cylc-run/u-dk655/share/fcm\_make\_pp/build/bin/transfer.py”, line 48, in **init** self.\_globus\_cli = nl\_transfer.globus\_cli AttributeError: ‘ReadNamelist’ object has no attribute ‘globus\_cli’ [FAIL] transfer.py \<\<‘ **STDIN** ’ [FAIL] [FAIL] ‘ **STDIN** ’ # return-code=1 2024-11-20T09:43:16Z CRITICAL - failed/EXIT Any suggestions? Cheers, James ---
**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\ **Post date:** [20 November 2024 12:22 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/35 "2024-11-20T12:22:41Z")
Hi James, I made some changes recently to the `postproc_2.4_archer2_jasmin_rewrite` branch in preparation for moving everyone to using Globus for data transfer. Please set the revision number of the branch to 5092 to pick up the version compatible with your current setup. Regards, Ros ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [20 November 2024 12:38 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/36 "2024-11-20T12:38:17Z")
Ok, thanks, Ros. Will this require a full stop and start with new completion of fcm\_make\_pp etc? James ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [20 November 2024 12:55 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/37 "2024-11-20T12:55:03Z")
Sorry, does this change only need to be done in fcm\_make\_pp or do I need to change the below in postproc/rose-app.conf too? meta=/home/n02/n02/ros/meta/postproc\_2.4\_archer2\_jasmin\_rewrite/rose-meta/archive\_and\_meaning/postproc/pp24\_t588 Thanks, James ---
**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\ **Post date:** [20 November 2024 16:18 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/38 "2024-11-20T16:18:58Z")
Hi James, Just in the `fcm_make_pp` app. And yes you will need to re-run `fcm_make_pp` & `fcm_make2_pp` tasks either by stopping the suite and starting again or reloading the suite and retriggering those tasks. Cheers, Ros. ---
**Author:** ![James\_Weber](https://avatars.discourse-cdn.com/v4/letter/j/e495f1/32.png) [@James\_Weber](https://cms-helpdesk.ncas.ac.uk/u/James_Weber)\ **Post date:** [21 November 2024 08:46 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/39 "2024-11-21T08:46:43Z")
Thanks, Ros. I made what I thought was the correct change but u-dk655 has failed again on pptransfer. Could you check my suite to see if I have implemented the revision correctly? Cheers, James ---
**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\ **Post date:** [21 November 2024 11:41 UTC](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539/40 "2024-11-21T11:41:18Z")
Hi James, As soon as ARCHER2/PUMA2 come back I’ll take a look. Regards, Ros. [Previous page](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539.md?page=1) [Next page](https://cms-helpdesk.ncas.ac.uk/t/vn13-3-amip-suite-on-archer2/1539.md?page=3)