Thanks, and I will make sure to keen on top of regularly cleaning up my PUMA space.
I started preliminarily analysing some of the output of the model, but I noticed that a couple of the files seem to be corrupted or not saved properly. Is this something that happens sometimes? For example, the file /work/n02/n02/tarlge/archive/u-cz110/19790501T0000Z/nemo_cz110o_1m_19790401-19790501_grid-T.nc. When I try to open the file in ncview on JASMIN it doesn’t recognise it as a NetCDF file, and the file size is small - implying something went wrong.
Is it possible for me to just re-insert the month and re-run this month in a bit-identical way?
Any help would be greatly appreciated!
Cheers,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
Is it okay for me to continue running the simulation forward, or would you recommend that I wait?
Cheers,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
It depends what you want. If those dodgey .nc files are important then you will need to rerun the suite from 1979-04 when Archer2 is back. You can’t just rerun a single cycle. If I remember correctly though, your suite only started in 1978-09, so if it was me, I’d probably just restart it from the beginning again.
Keep an eye on the files - if you see this happening again let us know.
Thanks, I just restarted as you suggested, and will keep an eye on it.
Although my two running suites u-da143 and u-da032 have delete_staged=true, the files from ARCHER2 don’t seem to delete automatically after they are transferred to JASMIN, is this feature working for other suites? Perhaps I need to update my suites if there were changes made to the GC5 suite after I took my copy?
Cheers,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
Sorry, I’ve just realised looking at the code that deleting staged data has not been modified to work with gridftp transfers. I’ll add this to my to-do. It’s not a trivial fix so will be a little while until this can work. For now you will have to manually delete the staged data.
No problem at all! Manual delete is fine for now, thanks for taking a look.
Cheers,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
I just realised that some of my new runs seem to have corrupted files in them. I don’t know whether this was a filesystem issue or an error on my part. Is there something that I could be doing wrong to have this effect?
The file I am looking at is on JASMIN but not ARCHER2 anymore:
It did happen at some point during the run while I was away that my ARCHER2 disk filled up, and then I had to retrigger the run as it had crashed. Is it possible that during this the files got corrupted? The model did continue to run without issue, so I thought it was okay.
Cheers,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
I have also been searching the documentation for information about restart files, but I am still not too sure which files are actually required to restart a run, I want to make sure that I save out some restart files for longer runs so that I can re-run parts if necessary, do you know how I can control this?
Many thanks,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
Thanks for this, I don’t seem to have this structure of postproc → NEMO anymore on PUMA2, should this be different?
I guess the thing I’m still confused about is the difference between output files and restart files. For example, are these files restart files? or are restart files saved somewhere else?
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
There are typically 4 start files needed for a coupled suite one each for the atmosphere and ice and two for the ocean – these files contain prognostic data. Output files contain diagnostic data and play no part in starting the model. All the files referred to above hold diagnostics.
I agree that the UM training would definitely be useful, unfortunately I just missed out on the last one as it was full, and I am away for fieldwork 16th November - 1st January so will miss out on the next one too. I have read through the online self-study notes and try to refer to them as much as possible.
Many thanks,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
This is what I was confused about, on the old PUMA I feel that there were more settings visible in the GUI under postproc. Perhaps I am doing something wrong here.
Thanks,
Tarkan
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
