# Ssh was working and has since stopped

**URL:** <https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698>\
**Category:** Uncategorized\
**Tags:** ARCHER2\
**Created:** [2 May 2025 11:38 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698 "2025-05-02T11:38:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![penmaher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/penmaher/32/437_2.png) [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)\
**Post date:** [2 May 2025 11:38 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/1 "2025-05-02T11:38:27Z")

</div>

Hi,

I have not run on puma2 for about 2 months. When I tried to run a previously working suite, I have been getting a series of errors. First, it was similar to the [error found here](https://cms-helpdesk.ncas.ac.uk/t/issues-with-submitting-coupled-task/534/3) about  
“modtime1 = os.stat(tmpfile).st\_mtime  
TypeError: coercing to Unicode: need string or buffer, int found”

So I started digging into it, and I can see my ssh-agent was not working. I followed the restarting agent step here but that did not fix it [11. Appendix B: SSH FAQs — NCAS Unified Model Introduction](https://ncas-cms.github.io/um-training/ssh-tasks.html#restarting-agent)

When I looked in my .ssh on archer2, my ssh files were missing. I have added them again. No idea why they were not there. I checked the steps on [setting up your ssh agent](https://ncas-cms.github.io/um-training/getting-setup-selfstudy.html) and everything looks okay.

When I run the suite on puma 2 I am getting the error

[FAIL] bash -ec H=$(rose\ host-select\ archer2);\ echo\ $H # return-code=1, stderr=  
[FAIL] [WARN] ln02: (ssh failed)  
[FAIL] [WARN] ln04: (ssh failed)  
[FAIL] [WARN] ln01: (ssh failed)  
[FAIL] [WARN] ln03: (ssh failed)  
[FAIL] [FAIL] No hosts selected.

So I can tell that puma2 can’t connect to archer2 to submit the job. Any idea what I broke while trying to fix my initial problem?

Penny

---

<div class="post-metadata">

**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\
**Post date:** [2 May 2025 12:33 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/2 "2025-05-02T12:33:31Z")

</div>

Hi Penny,

What do you get when you run `ssh ln03` on the PUMA2 command line?

Regards,  
Ros.

---

<div class="post-metadata">

**Author:** ![penmaher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/penmaher/32/437_2.png) [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)\
**Post date:** [2 May 2025 12:49 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/3 "2025-05-02T12:49:09Z")

</div>

Hi Ros,  
Thanks for picking this up. Much appreciated!

[penmaher@puma2 u-dm681]$ ssh ln03  
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @  
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  
Permissions 0640 for ‘/home/n02/n02/penmaher/.ssh/id\_rsa\_archer’ are too open.  
It is required that your private key files are NOT accessible by others.  
This private key will be ignored.  
Load key “/home/n02/n02/penmaher/.ssh/id\_rsa\_archer”: bad permissions  
penmaher@ln03’s password:

it then does not accept the password for the file that I know is correct.

---

<div class="post-metadata">

**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\
**Post date:** [2 May 2025 13:03 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/4 "2025-05-02T13:03:22Z")

</div>

Hi Penny,

It won’t accept the passphrase because the file permissions of your keys are too open, it then defaults to asking for you ARCHER2 password. Please change the permission on your `~/.ssh` directory and files:

`puma2 $ chmod g-r ~/.ssh`

Also make sure that the private keys within the .ssh directory are 600 pemissions (rw to only you).

Regards,  
Ros

---

<div class="post-metadata">

**Author:** ![penmaher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/penmaher/32/437_2.png) [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)\
**Post date:** [2 May 2025 13:15 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/5 "2025-05-02T13:15:49Z")

</div>

I have updated the permission. When I now type `ssh ln03` it asks for my pw, accepts it and logs into archer2

---

<div class="post-metadata">

**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\
**Post date:** [2 May 2025 13:18 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/6 "2025-05-02T13:18:25Z")

</div>

That’s great. All I think you need to do now is add the key to your ssh-agent.

`puma2 $ ssh-add ~/.ssh/id_rsa_archer`

Then you should be able to run `ssh ln03` and not be prompted for either passphrase or password.

Regards,  
Ros.

---

<div class="post-metadata">

**Author:** ![penmaher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/penmaher/32/437_2.png) [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)\
**Post date:** [2 May 2025 13:29 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/7 "2025-05-02T13:29:42Z")

</div>

Brilliant. Yes that fixed the first SSH issue.

I am now back to my original problem of running a suite, having it fail to run with the following error, which I believe is still a SSH error:

Traceback (most recent call last):  
File “/home4/home/n02-puma/fcm/metomi/cylc-7.8.12/bin/cylc-cat-log”, line 439, in   
main()  
File “/home4/home/n02-puma/fcm/metomi/cylc-7.8.12/bin/cylc-cat-log”, line 435, in main  
tmpfile\_edit(out, options.geditor)  
File “/home4/home/n02-puma/fcm/metomi/cylc-7.8.12/bin/cylc-cat-log”, line 265, in tmpfile\_edit  
modtime1 = os.stat(tmpfile).st\_mtime  
TypeError: coercing to Unicode: need string or buffer, int found

---

<div class="post-metadata">

**Author:** ![RosalynHatcher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/rosalynhatcher/32/11_2.png) [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)\
**Post date:** [2 May 2025 14:32 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/9 "2025-05-02T14:32:24Z")

</div>

Hi Penny,

In e.g. `~/cylc-run/u-dm681/log/job/20150101T0000Z/fcm_make2_um/04/job-activity.log` there is an error regarding the SLURM settings.

```auto
(ln03) 2025-05-02T13:55:58Z [STDERR] sbatch: error: AssocMaxCpuMinutesPerJobLimit
(ln03) 2025-05-02T13:55:58Z [STDERR] sbatch: error: Batch job submission failed: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)

```

There is no HPC resource for n02-NEW005875. We moved into a new accounting year on 1st April 2025 and there was no HPC resource application submitted for the GreenBlock project for Apr25-Mar26.

Regards,  
Ros.

---

<div class="post-metadata">

**Author:** ![penmaher](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/penmaher/32/437_2.png) [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)\
**Post date:** [6 May 2025 08:52 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/10 "2025-05-06T08:52:51Z")

</div>

Hi Ros,  
Thanks for the reply. I contacted Grenville directly to get the updated project code and now the suite is working as expected. Thank you for your help!  
Penny

---

<div class="post-metadata">

**Author:** ![system](https://europe1.discourse-cdn.com/flex013/uploads/cms_support/original/1X/1fd2411499ffcbc299fe756cd5cdf26e44956558.png) [@system](https://cms-helpdesk.ncas.ac.uk/u/system)\
**Post date:** [5 June 2025 08:53 UTC](https://cms-helpdesk.ncas.ac.uk/t/ssh-was-working-and-has-since-stopped/1698/11 "2025-06-05T08:53:48Z")

</div>

Closed on 5 June 2025 08:53 UTC

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
