Hi,
I’m having trouble with a DEST run (u-dq441) with regards to the output in the monthly mean UPM stream. I’m using a 3 month resubmission and the first month (Jan 2010) has sensible output but then for all subsequent months the output appears to be the fill value of about -1e9 and can’t be viewed with xconv.
The dump files look ok so I think the model is running fine, it is just an output issue. Have you seen anything like this before?
Cheers,
James
James
I can only see output for March 2010 in /home/n02/n02/jweber/cylc-run/u-dq441/share/data/History_Data/dq441a.pm2010mar - the fields I looked at with xconv look OK to me ?
Grenville
Hi Grenville,
Sorry, I should have said that the issue appears to involve the Section 34 stash, e.g. O3. Their fields have a single value of -1.0737e+09 and when I try to view the data in xconv the message “Erro: floating point value is Not a Number” appears.
Thanks,
James
Hi James,
Can I check where you got the DEST run from and whether you’ve been able to run it successfully before this.
Do you have any disk space issues or anything else that may prevent the files being written correctly? If those numbers were correct then tracer advection would fail, so if it’s still running and producing output and dumps I would assume it’s an output issue and not anything to do with the code itself.
I might suggest looking in other non-monthly mean files and seeing if the fields look OK there.
Best wishes,
Luke
Hi Luke,
I think this issue has resolved itself by my starting from the original December dump and just maintaining the 3 month resubmission and 90 day dump frequency.
I’ll do a longer run and let you know if there are any problems.
Best wishes,
James
Hi Luke,
I fear I spoke too soon! The suite, u-dq740, completed the first two resubmission of 3 months each (starting Dec 2009) and the 34001 pm output (dec, jan, feb) in the 20100301T0000Z archive file were fine. However, in the subsequent output file, 20100601T0000Z, the 34001 pm output has reverted to -1.0737e+09. I cleared out a lot of old files so there was plenty of space on my Archer2 work directory.
Is there a suite-specific limit on memory which I could be exceeding?
Best wishes,
James
Hi James,
Do you have diagnostic messages turned on, and to all pe streams? That can fill up disk pretty fast.
Otherwise I’m not sure what’s going on from the sound of this, sorry! Is it just the UKCA fields, or everything?
Best wishes,
Luke
Thanks, Luke. I’ve got
PRINT_STATUS=PrStatus_Min
RCF_PRINTSTATUS=PrStatus_Oper
It does appear to only be section 34 tracers which have the problem. Other stash outside of Section 34 (including Section 50) in a.pm are fine.
Could the header size for apm cause an issue?
Thanks,
James
Just an update, I ran a copy of u-dq740, u-dq848, with greatly reduced Section 34 a.pm output, header size increased from 60000 to 300000 for pp130 (which is linked to a.pm) and PRINT_STATUS and RCF_PRINTSTATUS both set to PrStatus_Min.
This returned sensible .pm fields for 34001 for Dec 2009 to Mar 2010 before returning -1.0737e+09 from April onwards, exactly the same as u-dq740. Other streams are still outputting sensible data many months after Mar 2010. Stash 51001 is also output in a.pm stream and suffers the same issue as 34001.
I will try adding a Section 34 diagnostic to another stream to see if it Section 34 that is the issue or the a.pm stream.
Best wishes,
James
Thanks James - I was about to ask if it looked OK in a different stream.
What UM version is this, and is it based on a UKESM configuration, and if so, which one?
I’m guessing, but maybe try not packing the data in the pm stream (go to model output streams, pp130, set packing to Unpacked)
Grenville
Thanks, both. I have set pp130 to Unpacked and will let you know what happens.
Luke, it is a vn11.1 suite and the rose-suite.info says: “THIS IS THE 1.0 RELEASE OF UKESM1.”
James
Hi James,
Just to confirm - do we know if Ewa’s original suite u-di800 managed to complete a long run? The ARCHER2 data seems to have been cleared, and pptransfer was turned Off (and pointing to someone else’s area on JASMIN anyway).
Hi Mohit,
That’s a good question. My original DEST attempts were based on a copy of u-di800 (Gill Thornhill’s u-dj323). The only DEST suite I know of which has produced long term data is u-ce185 which was run on Monsoon I think.
If Grenville’s suggestion doesn’t resolve this issue, I will look at using u-ce185 as a base.
Cheers,
James
James
The suite produces bad data at the end of the first cycle - look at O3 MASS MIXING.. in the start file and in dq740a.da20130601_00 - it’s a mystery how it runs at all.
Grenville
As expected, the packing change hasn’t resolved the issue. Will need to have a rethink.
James