I want to run an AMIP model for a short time – 16 or 15 months. I want it to produce a relatively small amount of output as PP data (so my post-processing can extract average values over the last 12 months of the simulation). I do not want the data archived to jasmin – it is small enough to stay on archer2 and having an archive step increases the risk of the workflow failing. (I plan to run this workflow order 50-100 times with each one being automatically generated from the base case).
I think I want archiving turned off which handles conversion of fieldsfiles to pp but turn of pptransfer which moves the data to jasmin. Correct?
One issue is how does the last month get converted? If there is no automatic way I think the way to proceed is to run in 1 month chunks and run on an additional month.
My current suite that is trying to do this is u-dr496.
many thanks
Simon
Hi Simon,
Noting that ARCHER2 /work disk is not for permanent storage of data, if you don’t wish to automatically transfer data to JASMIN simply turn off the pptransfer task.
As discussed in your other ticket, to automatically postprocess/archive the last month of data in a cylc 8 suite, you currently need to run one extra cycle.
Regards,
Ros.
Hi Ros,
I did try and have pptransfer in my short job. Just had a whole bunch of globus transfer fail which stops the job running and required a bunch of manual interventions. My goal is to have 10s of jobs running so doing manual intervention for that many will take all my time. For now, I will not have pptransfer running and just manually transfer everything over to jasmin using globus cli and clean using cylc (or rm -rf !). The run size will be, I think, circa 25 Gbytes/run of which circa 750 Mbytes are the pp data I actually want!
Does that seem reasonable?
Simon