# PiControl HadGEM3-GC3.1 suite
**URL:** https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636
**Category:** Uncategorized
**Created:** [30 January 2025 14:14 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636 "2025-01-30T14:14:48Z")
**Posts on this page:** 17
**Page:** 2
### Author:  [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)
#### Post date: [12 February 2025 12:39 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/21 "2025-02-12T12:39:37Z")
Hi Penny,
Thanks. `/hpc/common_xcs/um1` is the Met Office equivalent to our `$UMDIR` under `/work/y07/shared/umshared`. So the ORCA1 orography file is already on ARCHER2 at:
`/work/y07/shared/umshared/ancil/atmos/n96e/orca1/orography/globe30/v6`
Regards,
Ros
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [13 February 2025 10:39 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/22 "2025-02-13T10:39:39Z")
Thanks Ros.
I have added the orog files. Thanks for the links. I have run the suite for a month from a cold start (i.e. not using a start-dump as a spin up) for an 1850 start date as a test to see if the ozone redistribution is working. My ozone output at the end of the month has no banding. Picture
I have two thoughts:
1. The ozone redistribution is not working, or
2. The ozone redistribution is not supposed to do anything for 1850 conditions.
Do you know if the ozone redistribution should show anything after 1 month of simulation for 1850 conditions?
I am working on suite u-dm681 on the branch piControl\_ozone\_redistribution
My plan is to run a future scenario of this suite but for now I am just trying to get each bit working one thing at a time to keep things as simple as possible. Ideally I want to get the ozone redistribution working before moving onto the next task of converting the suite from piControl to SSP245.
Thanks,
Penny
---
### Author:  [@AnnetteOsprey](https://cms-helpdesk.ncas.ac.uk/u/AnnetteOsprey)
#### Post date: [13 February 2025 11:13 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/23 "2025-02-13T11:13:17Z")
Hi Penny,
If you are following our instructions, then the ozone redistribution doesn’t do anything in the first year. The is because it needs data from the previous 1 or 2 years of the simulation to work out the redistribution. It should be possible to specify ozone data from a previous spinup run, but we haven’t implemented that in our example suites.
Annette
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [14 February 2025 11:20 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/25 "2025-02-14T11:20:30Z")
Morning,
My ozone redistribution code is not yet up and running. I have changed my suite from piControl over to SSP245. So there is likely a few things broken which I am working on fixing. When I run the suite, the retrieve\_ozone step is failing with the error:
[FAIL] file:/work/n02/n02/penmaher/cylc-run/u-dm681/share/data/ozone\_redistribution/mmro3\_monthly\_CMIP6\_2014\_N96\_dm681-ancil\_2anc=source=/home/n02/n02/um1/cmip6/ancils/n96e/ssp245/Ozone/v1/historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc: [Errno 13] Permission denied: ‘/home/n02/n02/um1/cmip6/ancils/n96e/ssp245/Ozone/v1/historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc’
2025-02-14T09:11:08Z CRITICAL - failed/EXIT
I can see I have some problems with paths to files. I am in a little bit of a muddle about the differences between .anc files and files of the form mmro3\_monthly\_CMIP6\_\*\_edited-ancil\_2anc which is making it hard to get the correct paths for $OZONE\_INITIAL, $OZONE\_ANCIL etc.
I have seen in the docs, that it is recommended that I split the data into yearly fields to speed the code up. Happy to do that. But not sure which file yet is the right one to split or where to put it.
For an SSP245 run starting in 2015 I assume I want the following files as input ozone:
```
/work/y07/shared/umshared/cmip6/ancils/n96e/ssp245/Ozone/v1/historic_interpolated_3d_ozone_n96e_2015_2099_ants.anc
```
So I have set in u-dm681/rose-suite.conf
OZONE\_INITIAL=‘$UMDIR/cmip6/ancils/n96e/ssp245/Ozone/v1/historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc’
In u-dm681/ozone-redistribution.rc I have
OZONE\_ANCIL = ‘mmro3\_monthly\_CMIP6\_$(rose date -c -f %Y)_N{{“216” if “216” in ANCIL\_OPT\_KEYS else “96”}}_${RUNID}-ancil\_2anc’
I currently have year splitting off but the error message above tells me that $OZONE\_ANCIL is still used, so I plan to split the data now.
For the $OZONE\_ANCIL do I need data moved over onto archer in the following location:
/work/y07/shared/umshared/cmip6/ancils/n96e/timeseries\_2014-2100/OzoneConc/v1/
or do I only need the 2014 data to get started, i.e.
/work/y07/shared/umshared/cmip6/ancils/n96e/timeslice\_2014/OzoneConc/v1/mmro3\_monthly\_CMIP6\_2014\_N96\_edited-ancil\_2anc
Thanks in advance!
Penny
---
### Author:  [@AnnetteOsprey](https://cms-helpdesk.ncas.ac.uk/u/AnnetteOsprey)
#### Post date: [14 February 2025 14:48 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/26 "2025-02-14T14:48:28Z")
Hi Penny,
I will do my best to answer your questions, but as you’ve seen the OR scheme is quite complicated, and I can’t claim to understand it all.
You are correct to set `OZONE_INITIAL` in `rose-suite.conf`. The problem is that the `$UMDIR` variable is being evaluated on puma2 as `/home/n02/n02/um1`. I think you should be able to escape the variable so use `'\$UMDIR'` instead, and it will evaluate the variable on Archer2. If that doesn’t work you will need to just use the full path which is `/work/y07/shared/umshared`.
The `mmro3...` files are the name given to the ozone ancillaries for each year. The `retrieve_ozone` app makes a symlink to whatever the original ozone ancillary file is in `share/data/ozone_redistribution`. How it works though is that the previous year (in your case 2014) points to the following year (so 2015). I don’t know why it does this, but it doesn’t matter if the source ancil is split or not. So you shouldn’t need the 2014 ancil here.
The `redistribute_ozone` code then uses that original ancil plus the diagnostics from the previous 2 years of the model to generate a new ancil for the model to use. This will also be named `mmro3` but with the correct year and lives in `share/data/etc/ozone`. In the first year with no other source data it will just use the orginal ancil. Does that make sense?
Annette
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [17 February 2025 09:44 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/27 "2025-02-17T09:44:40Z")
Thanks for the explanation Annette. That is very helpful. One point of confusion I have is, what is the difference between the ozone ancil files stored in
/work/y07/shared/umshared/cmip6/ancils/n96e/_SOME\_SSP\_SCENARIO_/historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc
and the ozone ancils files stored in:
/work/y07/shared/umshared/cmip6/ancils/n96e/timeseries\__SOME\_YEAR\_RANGE_/OzoneConc/v1/mmro3\_monthly\_CMIP6\_1849-2015\_N96\_edited-ancil\_2anc
The files of the form historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc are in SSP scenario files, and from the file name and path name combo, it is pretty clear these are the projected ozone input files needed for a future scenario run. In a suite I am trying to recreate, this the file they use, so I plan to use it too.
The files of the form mmro3\_monthly\_CMIP6\*\_2anc are clearly ozone input files too. Are these for piControl runs or what other context? As the files names are rather different, I want to make sure I understand their differences.
In addition, if I plan to use historic\_interpolated\_3d\_ozone\_n96e\_2015\_2099\_ants.anc is it stil recommended I split the years up? If yes, could you suggest how best to do this please? Wondering if this is in xconv or is there another preferred method?
Thank you!
Penny
---
### Author:  [@AnnetteOsprey](https://cms-helpdesk.ncas.ac.uk/u/AnnetteOsprey)
#### Post date: [18 February 2025 08:57 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/28 "2025-02-18T08:57:03Z")
Hi Penny,
There is documentation for the CMIP6 ancils, which includes details of the ozone files:
[https://code.metoffice.gov.uk/trac/ancil/wiki/CMIP6/ForcingData](https://code.metoffice.gov.uk/trac/ancil/wiki/CMIP6/ForcingData)
Whether or not to split the files into years depends on how long it takes to read in the ozone ancillary in iris. You could try that and see how long it takes. Anyway I will look out the code.
Annette
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [24 February 2025 10:01 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/29 "2025-02-24T10:01:41Z")
That was exactly what I needed. Thanks Annette.
Could I have some further help please? My suite is failing in recon with the error:
???!!!???!!!???!!!???!!!???!!! ERROR ???!!!???!!!???!!!???!!!???!!!
? Error code: 1
? Error from routine: io:file\_open
? Error message: Failed to open file /work/y07/shared/umshared/ukcmip6/ssp585\_N96O1\_ensemble1\_dumps/bg466a.da20150101\_00
? Error from processor: 0
? Error number: 2
The filename is assigned in app/um/rose-app.conf where
astart=‘$UMDIR/ukcmip6/ssp585\_N96O1\_ensemble1\_dumps/bg466a.da20150101\_00’
This is the correct path on archer. I have read access to the file and I am open it in xconv and look at the fields. Any ideas on why the file can’t be opened? Thank you in advance.
Penny
---
### Author:  [@grenville](https://cms-helpdesk.ncas.ac.uk/u/grenville)
#### Post date: [24 February 2025 11:09 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/30 "2025-02-24T11:09:10Z")
Hi Penny
that’s odd – please try copying the file(s) to somewhere in your /work space & point the suite to them.
Grenville
---
### Author:  [@mdalvi](https://cms-helpdesk.ncas.ac.uk/u/mdalvi)
#### Post date: [24 February 2025 11:20 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/31 "2025-02-24T11:20:45Z")
> [@penmaher](#):
>
> astart=‘$UMDIR/ukcmip6/ssp585\_N96O1\_ensemble1\_dumps/bg466a.da20150101\_00’
astart=‘$UMDIR/ukcmip6/ssp585\_N96O1\_ensemble1\_dumps/bg466a.da20150101\_00’
If a central file is specified as ASTART Reconfiguration is likely not required as this will try to overwrite the file (which might be the actual error here- no permission to open the file as read/write).
If you need to run Reconfiguration, specify the above file as AINITIAL= and set
astart=$DATAM/$RUNID.astart
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [28 February 2025 11:18 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/32 "2025-02-28T11:18:03Z")
Thanks mdalvi and Granville. I tried both options, using ANITIAL did not work but moving the files did.
I have managed to make further progress, but I am stuck at the moment as I am not getting any meaningful error message.
The model is failing during the coupling phase. I am expecting the problems are in the UKCA part of the code/input files issue.
In the job.err I get the usual warnings and then the job does an MPI\_abort. Any idea how I work out why it aborted? I can’t find anything usful in the cycl dir to tell me what is going on.
the run is here is anyone would be able to help:
/home/n02/n02/penmaher/cylc-run/u-dm681
---
### Author:  [@mdalvi](https://cms-helpdesk.ncas.ac.uk/u/mdalvi)
#### Post date: [28 February 2025 11:54 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/33 "2025-02-28T11:54:07Z")
Hi Penny,
The actual model log files are found in work/date-time/coupled as _pe\_output/.fort6.peXX_ for UM, _ocean.output_ for NEMO and _debug.root.XX_ for CICE.
In your case it is the ocean model NEMO that has aborted as seen in the ocean.output:
```auto
===>>> : E R R O R
===========
stpctl: the zonal velocity is larger than 20 m/s
======
kt= 3 max abs(U): 5190. , i j k: 171 82 1
output of last fields in numwso
===>>> : E R R O R
===========
step: indic < 0
dia_wri_state : single instantaneous ocean state
~~~~~~~~~~~~~ and forcing fields file created
and named :output.abort .nc
===>>> : E R R O R
===========
```
I am not familiar with NEMO, but an error this early indicates something in the model settings and initial data.
Mohit
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [28 February 2025 11:57 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/34 "2025-02-28T11:57:40Z")
I was confused why I was not seeing the coupled folder ( I was on puma looking in cylc and should have been on archer). Thanks for the pointer to the problem. Much appreciated!
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [4 March 2025 13:35 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/35 "2025-03-04T13:35:19Z")
Hi Ros,
Is there any chance there is another system at the MO that the files in /data/d01/ukcmip6/ssp585\_N96O1\_ensemble1\_dumps/ might be located on?
One of these input files is causing NEMO to crash and I suspect it is the ssp585\_N96O1\_ensemble1\_dumps/bg466i.restart.2015-01-01-00000.nc file.
This is the correct filename used in the CMIP6 runs, but I am wondering if the file I am using is not the same as the CMIP6 production runs (older revision or similar).
Penny
---
### Author:  [@RosalynHatcher](https://cms-helpdesk.ncas.ac.uk/u/RosalynHatcher)
#### Post date: [4 March 2025 15:13 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/36 "2025-03-04T15:13:19Z")
Hi Penny,
I’m sorry, but the MO old HPC (xce/xcf) is the only location I have found those files. I did just checksum them to make sure what we have on ARCHER2 is the same.
I’m not sure what else to suggest at the moment.
Regards,
Ros
---
### Author:  [@penmaher](https://cms-helpdesk.ncas.ac.uk/u/penmaher)
#### Post date: [4 March 2025 15:31 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/37 "2025-03-04T15:31:16Z")
That is helpful Ros thank you. I am going to the Met Office on Thursday to ask around and see if anyone can help.
---
### Author:  [@system](https://cms-helpdesk.ncas.ac.uk/u/system)
#### Post date: [3 April 2025 15:31 UTC](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636/38 "2025-04-03T15:31:17Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
[Previous page](https://cms-helpdesk.ncas.ac.uk/t/picontrol-hadgem3-gc3-1-suite/1636.md?page=1)