Installing glm startdata repeatedly fails

Hi,

The task install_glm_startdata regularly (I estimate 50% of the time) fails on most simulations I run. If the task succeeds, it typically takes ~10 minutes. If the task takes longer than this, then it will stall and eventually time out. This considerably prolongs the time taken to complete jobs. I think the task is just a “moo get” command to pull the start dump from MASS. Have you come across this problem before? I am using Monsoon.

Thanks, Andy

Hi Andy

This is a question for the Monsoon team - I have forwarded you query. Could you not download the file once and subsequently read from disk (as a work around.)

Grenvillle

Hi Andy, did you ever get this resolved? I’m having the exact same issue.

I’ve already queried this with the Monsoon team and have tried extending the wall-time limit to 3 hours, but it gets the majority of the way there and then runs out of time.

I was told 3 hours is the maximum for this step. I’ve also wondered about manually retrieving the data but I haven’t had much guidance on that.

What did you do in the end?

Amethyst

(ETA: I’ve also tried halving the number of ensemble members because sometimes a deterministic will get through within a matter of minutes, but this hasn’t worked.)