I’m working on porting the RAS suite to ARCHER2, making use of the ants singularity container. The working suite for this is u-cq149, and the trac ticket is 338.
For the ancil_topographic_index step I need a topographic_index.nc input file. Is there already a suitable file available on ARCHER2 that I can use, or do I need to get this from the Met Office?
thanks,
Doug
Hi Doug
That file is not on Monsoon as far as I can see.
it may need to come from /data/users/hadzc/RMED/ANTS
Grenville
Doug
Copied topographic_index.nc to /work/y07/shared/umshared/ancil/atmos/master/soil_data/hwsd/v4
Grenville
Doug
Just to let you know - this isn’t the right place to put topographic_index.nc – I’m not sure where is at the moment, so it may not remain here.
Grenville
hi Grenville,
Thanks for this. I’ll try with the file in this location - but will make a note it might move in future.
cheers,
Doug
hi Grenville,
I’m getting this error when I try accessing the topography netcdf file:
OSError: [Errno -101] NetCDF: HDF error: b'/work/y07/shared/umshared/ancil/atmos/master/soil_data/hwsd/v4/topographic_index.nc'
Exception ignored in: <function CFReader.__del__ at 0x2acd8c5db290>
Traceback (most recent call last):
File "/opt/conda/envs/ants/lib/python3.7/site-packages/iris/fileformats/cf.py", line 1129, in __del__
self._dataset.close()
AttributeError: 'CFReader' object has no attribute '_dataset'
Could you check the netcdf file? Ncview won’t open the file for me either, so I wonder if it might be corrupted?
thanks,
Doug
Hi
I’m also in need of this file (adding ANTs ancillary generation for RA3 suite driven by ERA5) any luck in finding a working version? I get the same error as Doug for my qrprarm.hydtop generation. I also can’t view in xconv and ncdump returns unable to open file errors
Never mind, I found a working copy in Doug’s file space (thanks Doug!). The last piece of the puzzle is getting the SRTM orography generation to work in my suite now is this working in your suite Doug?
Hi all,
Yes - Claudio Sanchez shared his topography file with me (which I just moved across to ARCHER2 today 
). I’ve asked if he’s okay with this file being put in the shared data space as well, so it’s more accessible for everyone.
He is also asking the data owners of the SRTM orography data if he can share this with me. If we get a positive answer from them, I’ll ask if these also could be put into the shared data space here as well.
cheers,
Doug
Apologies Doug – this one slipped through the net – glad you got the file!
Ah great, I’ll keep an eye out.
@grenville on shared space note, is there a better ants_python_path to use than my own anaconda ants installation path?
There is a container with ANTS which you can use at:
$UMDIR/bin/python_ants
This may help
thanks, I’ll give it a whirl. Makes the suite more shareable if I don’t have pointers to user space in it so I’ll test if everything still works smoothly pointing to that instead!
@douglowe I tried giving your suite a run yesterday - it looks like you’ve done a lot, which is great. Is it finished, or are you still looking for the file: SRTM_concat.nc in Regn1_resn_1_ancil_mean_orog ? I was stuck here yesterday, with just a handful of apps left to run.
I admit that I pinched a few other files, so I mention this in case you or Helen are looking for feedback when making things available to others.
Files: c4_percent_1d.nc , qrparm.orog , topographic_index.nc
Scripts: ancil_lct_postproc_moruses.py
Directories: ancil_master_cap
Executable: central_ancillary.exe (This is available at $UMDIR/CAP9.1/build/bin/central_ancillary.exe if you want to use this centrally installed version)
I don’t know if this information is helpful, but I hope it may be. Thanks again, Dave
hi Dave - I’m still waiting for the SRTM files from Claudio, but other than that I think we’re close to being sorted (for this iteration of the tool set, at least).
Feedback on what user actions are needed to enable the workflow is very helpful - I realise there’s a lot of fudges I’ve made to try to get this to work, a number of which I’ve already forgotten about, so being reminded of these is very helpful
Regarding the ancil_lct_postproc_moruses.py script. This isn’t in the ANTS container yet - Claudio has said that this (and another script?) aren’t likely to make it into the ANTS release quite yet, but they are being used still in the workflow. Would it be possible for these extra scripts in be included in the next iteration of the ANTS container, or should we download these separately still?
I’ve setup my workflow (suite u-cq149) to use the ANTS container - I modified the python_env script to do this. The one step which I’ve not got working yet is the initial download of the extra scripts from the FCM server.
I would probably suggest not modifying the container, but we could install them somewhere and then provide the full path to the running script (ie replace ancil_lct_postproc_moruses.py with [/work/some location]/ancil_lct_postproc_moruses.py) ?
I will ask my colleagues about shared spaces on Archer, as we aren’t supporting this and it seems like a temporary approach until ANTS gets updated. If you think that this is a better solution than downloading extra scripts in the suite, then let me know.
Having a shared space for these scripts would be a better solution than downloading the scripts from the suite itself I think - as the combined download is near ~1Gb, for the sake of only 1 or 2 scripts. Plus this is the one step in the process I’m not sure how to include at the moment - it needs fcm for the download, which I guess could be run on puma, and then the relevant files copied to archer2 during setup - but adapting the workflow to do this would require a reorganising of the step order (I think?), which would be quite disruptive.
The two sets of extra code that are downloaded are:
ANTS_FCM_URL=fcm:ancil_ants.xm_br/dev/claudiosanchez/r10519_RAS_ants_v0p16p0@10623
CONTRIB_FCM_URL=fcm:ancil_contrib.xm_br/dev/claudiosanchez/r10581_RAS_contrib_v0p16p0@10872
I think we probably only need the CONTRIB pack though.
@douglowe any word on those SRTM files and if they can be shared?
Also @dcase I’m talking with Paul Field at the metoffice about sharing my RA3 nesting suite updates to feedback into the standard RA3 nesting ARCHER2 suite, it would be great to have that topographic_index.nc file not in a user space before sending the updates.
Cheers
Helen
hi Helen,
Unfortunately I’ve heard nothing back on the SRTM files yet. I’ll chase this up again. I do also need confirmation that the topographic_index.nc file can be shared more widely (I’ll chase this in the same email too).
cheers,
Doug