I have a newly copied suite (u-dh833) taken from Monsoon to run on ARCHER2. I want to transfer the outputs to JAMSIN, but not sure how to do it properly. Below are a few things that I thought I need to change, please check:
I have changed “archive_root_path” under “Archer Archiving” to “/$ROSE_DATAC”
I have no ide what to change in the “transfer_dir” under JASMIN Transfer from the original line “/gws/nopw/j04/ukca_vol1/nlabraham/archive”.
Any other changes needed? Thanks!
First you need to change the version of the postproc branch. The one you are using is out of date and won’t work. Looks like this suite is using optional configuration overrides so you’ll need to change this in the app/fcm_make_pp/opt/rose-app-archer2.conf file: Change the version of the postproc_2.3_pptransfer_gridftp_nopw branch from 4422 to 5095.
You will need to re-run the suite, to pick up the changes and rebuild the postproc code.
Set archive_root_path to$ROSE_DATAC - no need for the leading /
transfer_dir is the location where you want to put the data on JASMIN. E.g. A GWS to which you have access.
Regarding point 3, I am not sure what you meant exactly, as I do not know what GWS referes to. After I checked old archived data from previous runs, it seems some model outputs were archived under /work/n02/n02/emxin/cylc-run/u-xxxxx/share/data/History_data. is it the right place to store data? I do not see an “archive” directory there.
In addition, how about using this path: /gws/nopw/j04/ukca_vol1/emxin/archive, is it working? Please help. Thanks
JASMIN GWS stands for JASMIN Group Workspace. In order to use any of the JASMIN group workspaces you have to be a member of the project. You can put data under /gws/nopw/j04/ukca_vol1 assuming you have access to the the UKCA GWS. I don’t know which projects you are a member of, in order to advise where to put your data.
I previously used ukca_vol1 as our output on JASMIN, and it seemed working well, but since I moved to /gws/nopw/j04/bas_climate last week, I found I could not find the model output anywhere, which is very strange. This happened after I re-upload my SSH-key to the my JASMIN account as I could not log into it.
Since then, I changed job JASMIN Transfer_dir from old /gws/nopw/j04/ukca_vol1/nlabraham/archive to a new directory /gws/nopw/j04/bas_climate/users/xinyang/archive (eg in u-dk569). However, then I could not find any of the model outputs (old and new) on JAMSIN (neither in /home nor in sci-vm-01.jasmin.ac.uk). Maybe I have to move back to ukca_col1? Please help. Thanks!
Best,
Xin
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
You suite u-dk569 currently has transfer_dir set to /gws/nopw/j04 ?? It looks like you’ve re-run the job since the above as there are no log files for the pptransfer task so I cannot diagnose what has happened. I also do not have access to bas_climate GWS.
Please reset the transfer_dir to /gws/nopw/j04/bas_climate/users/xinyang/archive and re-run the suite. Let me know once the pptransfer task has run and then I can take a look at the log files and see where it is transferring the data to.
Strangely, I could not add bas_climate to the Transfer_dir box in Rose. I submitted several jobs, however, after I re-opened the saved suites, I found Transfer_dir was unchanged(or unsaved), showing “/gws/nowp/j04” even I added the second part “/bas_climate/users/xinyang/archive” to it. But it was gone/missed. I do not know what happened. Maybe I am not allowed to use this space? BTW, my JASMIN account name is “xyang” but my puma and ARCHER2 account is emxin. Will this conflict cause the problem?
Best,
Xin
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
I don’t know why you can no longer edit that box. Can you edit others? I would suggest logging out of PUMA2 and back in again.
If you still can’t edit it, you can change the suite files direct. You will find the transfer_dir variable in the file ~/roses/<suiteid>/app/postproc/rose-app.conf.
It doesn’t matter that your JASMIN & ARCHER2/PUMA2 usernames are different. Gridftp handles this when you set up the short-lived credentials certificate.