Hi,
The task install_glm_startdata regularly (I estimate 50% of the time) fails on most simulations I run. If the task succeeds, it typically takes ~10 minutes. If the task takes longer than this, then it will stall and eventually time out. This considerably prolongs the time taken to complete jobs. I think the task is just a “moo get” command to pull the start dump from MASS. Have you come across this problem before? I am using Monsoon.
Thanks, Andy
Hi Andy
This is a question for the Monsoon team - I have forwarded you query. Could you not download the file once and subsequently read from disk (as a work around.)
Grenvillle
Hi Andy, did you ever get this resolved? I’m having the exact same issue.
I’ve already queried this with the Monsoon team and have tried extending the wall-time limit to 3 hours, but it gets the majority of the way there and then runs out of time.
I was told 3 hours is the maximum for this step. I’ve also wondered about manually retrieving the data but I haven’t had much guidance on that.
What did you do in the end?
Amethyst
(ETA: I’ve also tried halving the number of ensemble members because sometimes a deterministic will get through within a matter of minutes, but this hasn’t worked.)
Hi Andy, Amethyst, Grenville,
This has started happening also to me in the last few days, in most of the simulations I’m running (various suites based on the regional nested suite, vn 12.0). So far I haven’t been able to work out why some times it does work and the other times it doesn’t.
I can of course download the file so this is not the most urgent query, but it would be nice to know what the problem with install_glm_startdata is.
Many thanks for your support
Ambrogio
Hi Ambrogio,
We have not received any further update from the Monsoon team.
If Andy/Amethyst have found a solution, hopefully they might be able to update here, otherwise I can only suggest contacting the Monsoon team. (monsoon AT metoffice.gov.uk)
Regards,
Ros.