Future suite finished, some files left in History_Data

Unified Model
,
1

Hi CMS Helpdesk:
My first SSP585 future photosynthesis/acclimation run from 20150101-20510101 has finished successfully (suite ID u-cm923)! Thanks for the prior help with finishing the historical run from 1949-20150101. But the December 2050 data and the 20510101 dump data are missing in the ARCHER2 archive directory and on JASMIN. I see that data in the /work/n02/n02/pmcguire/cylc-run/u-cm923/share/data/History_Data directory.

Maybe I didn’t configure the suite properly or something. What is the best way to get the rest of the data from History_Data directory to the more-permanent archive directory on JASMIN? Yes, I can just figure out how to scp or sftp it, but maybe there’s a more proper way, like running an extra cycle or an extra postproc/pptransfer app or something.
Patrick

2

Patrick

set ARCHIVE_FINAL=true & try re-triggering the postproc task – I’m not 100% sure this will work.

Grenville

3

Thanks, Grenville:
I just tried to do rose suite-run --restart for u-cm922. But when I do this, there are no apps in the cycle GUI and there is no postproc app in that GUI window. It is an empty window. The suite previously finished successfully. Any suggestions?
Patrick

4

insert the postproc task maybe (bit of a guess again)? - it’s a pain how this final cycle stuff works.

5

Hi Patrick,

You’ll need to re-insert the postproc task for the final cycle, as Grenville says. As the suite has finished it’s a little bit of a pain.

I don’t think a simple cylc insert <suiteid> postproc.<cycle point> will work with the empty GUI but do try just in case.

Otherwise the easiest way is to extend the run out for one more cycle. Then do

rose suite-run --restart -- --hold so it starts the suite up but doesn’t run the next cycle.

Then re-insert the postproc task to the previous cycle with:

cylc insert <suiteid> postproc.<cycle point>

Cheers,
Ros.

6

Thanks, Grenville & Ros:
I just tried a rose suite-run -- --warm 20490701T0000Z for u-cm922, to restart the last cycle. And I have set ARCHIVE_FINAL=true in postproc , and then a rose suite-run --reload. And then I retriggered postproc.

Hopefully, this will get the files for December 2050 and the 20510101 dump file. I am not sure if it will, since they are after the SON (September October November) season of the calendar year. For the Autoassess from 2016-2050, it actually goes from December 2015 - November 2050, so I don’t need the December 2050 data for that, but It would be good to archive it and the 20510101 dump on JASMIN.
Patrick

7

Hi Grenville & Ros:
The postproc hack seemed to work! The extra end-of-the-year files are now in /work/n02/n02/pmcguire/archive/u-cm922/20490701T0000Z/ .

So, now I have retriggered the pptransfer app, which had already run successfully for the files up to 20501201`. I wonder if it will just try to transfer the extra end-of-the-year files to JASMIN, or if it will transfer the whole cycle again?

Patrick

9

Hi Grenville & Ros:
Just to clarify: The --warm start does run the whole cycle again. I did stop/kill the model before it was run again, and I just triggered and ran the postproc and pptransfer apps.
Patrick

10

Hi Grenville & Ros:
And to answer the question that I posed, it looks like pptransfer just transferred the 12 files at the end of the year 2050 with the dump at the beginning of the year 2051. This is what I wanted. So it worked!

These are the files transferred, as listed on ARCHER2 in ~pmcguire/cylc-run/u-cm922/log/job/20490701T0000Z/pptransfer/01/job.out:

cm922a.da20510101_00
cm922a.pa2050dec.pp
cm922a.pb20501001.pp
cm922a.pc20501001.pp
cm922a.pd2050dec.pp
cm922a.pe2050dec.pp
cm922a.pf20501001.pp
cm922a.pg20501001.pp
cm922a.pi20501001.pp
cm922a.pj20501001.pp
cm922a.pk2050dec.pp
cm922a.pl2050dec.pp

Patrick

11

Yes, pptransfer you’re using is running rsync so will simply update any changed files and transfer any new files.