ECMWF have just released CDS-beta for downloading model level ERA5 data (necessary for driving the UM with ERA5). I’ve got it working and changed all my scripts to work with cdsapi v 0.7.0 but I now realised my suites are using an install included in cray-python on archer, rather than my own conda environment (I remember trying to get suites to use my own conda environments previously and it being horrific).
Any chance you can update the cdsapi install to the latest version in cray-python?
If you wanted things faster, then you can install it yourself. I know you’ve tried this - but it is on conda, and there are more docs How to use the CDS API | Copernicus Climate Data Store (and related) so may have improved since your last attempt??
I’m not sure if we can do more??? Sorry,
Dave
P.S. Others will read this and may be able to contribute a more helpful answer
Thanks Dave. I’ve emailed archer support so will see what they say.
My previous attempts to use different conda environments for different UM modules on archer were a complete nightmare to be honest - if you have a clever way of doing that I’m very happy to look into using my existing conda venvs though!
e.g. if I add a conda activate / deactivate into the bin/main.sh routine in an app (e.g. u-da699/app/get_era_data) it comes back with an init error as here: Install of convsh on archer2? - #18 by shakka
I’m not entirely sure if this is helpful, but I would usually try to put environment loading into the cylc prescript (if you’re using rose/stem). Then run whatever you want as the app.
I seem to recall conflicts with what you had in your .bash_profile (or.bashrc) – you should be able to set up an environment exactly as needed (you may need to modify .bash files to avoid conflicts) with the pre-script.
After a back and forth with archer, I removed some of the conda-sepcific initialisation stuff from my .bashrc and then updated the suite.rc to remove the reference to cray-python, and instead call my own conda environments using ‘source activate’ instead of ‘conda activate’ in the pre-script. That seems to have worked - worth noting for future!
