FCM make fail for global suite u-bb316

Hello,
I’m getting the following job.err message

‘Environment variables set for netCDF Fortran bindings in
/apps/libs/netCDF/intel14/fortran/4.2/
You will also need to link your code to a compatible netCDF C library in
/apps/libs/netCDF/intel14/4.3.2/’

when I try to run a copy of the u-bb316 global suite. Any ideas how I can resolve this?
Many thanks,
Natalie

Hi Natalie:
What is your JASMIN username and is the suite number for the suite you’re working on the same as the u-bb316 global suite?

I have been running a modified copy of the u-bb316 global suite (it is ~pmcguire/roses/u-da046 ). I don’t remember if it was decided that I would run the global suite or you would. But I can stop if you were the chosen one.
Or I can keep running it, if I was the chosen one.

I vaguely recall seeing warnings in other suites like you are quoting here, and I never worried about those warnings. But I can’t immediately find a suite that had those warnings. Maybe you don’t need to worry about those warnings? But I don’t see those warnings in my u-da046 global suite that is currently running. So, you can use diff -r to compare your copy of the u-bb316 suite with my ~pmcguire/roses/u-da046 suite, to see what the differences are, and maybe that will fix the warning messages that you see?
Patrick

Hello Patrick,
I am not the chosen one this time - lol.
I am, however, looking for a simple low-res suite to do a global run on JULES and I remember you mentioned u-bb316 before. Is your suite u-da046 the best one for this purpose?
Perhaps the errors I was getting aren’t the real issue, in that case, as the suite failed at fcm make. It copied to my roses folder as u-dd217.
Natalie

It looks like the error messages for fcm_make in your u-dd217 suite are the ones that begin after the lines of the job.err file that you quoted.

The error messages begin with [FAIL] mpif90 -oo/water_constants_mod.o -c -DSCMA -DBL_DIAG_HACK -DINTEL_FORTRAN -I./include -I/apps/sw/libs/netCDF/intel14/fortran/4.2/include -heap-arrays -fp-model precise -traceback /home/users/ndouglas/cylc-run/u-dd217/share/fcm_make/preprocess/src/jules/src/params/standalone/water_constants_mod_jls.F90: command not found.

The key word here is the word [FAIL]. That often means an error instead of just a warning.
The lines before that don’t have this [FAIL] keyword.

I think that this means that for whatever reason, it couldn’t find the compiler mpif90.

Maybe you didn’t go through this tutorial for how to modify u-bb316 and then run it on JASMIN?:

That tutorial might need a bit of updating, but it worked in October. Let me know if you see anything that needs changing.
Patrick

Hi Patrick,
I didn’t know about this tutorial, thank you! I have checked out a copy of u-as052 and made the changes - I’m just waiting for access to the ncas_generic workspace before running the suite.
FYI, regarding tutorial point 9, the original suite has output_dir='/work/scratch/pmcguire/config/outputs' and not
output_dir='/work/scratch-pw2/pmcguire/config/outputs'. (pw2 IS MISSING)
and I was little unclear if point 11 actually required any changes to the suite. I hope that’s helpful to you.
I will let you know how it goes, thanks again!
Natalie

Hi Natalie:
Hopefully you’ll get approved for ncas_generic GWS access soon.
I’ve updated steps 9 & 11 of the tutorial. Thanks for pointing those issues out to me.
The tutorial might need further updating. Please do let me know if you see any other issues with it.
Patrick

Hi Patrick,
I finally completed the global run with the u-as052 suite and made a cool video to compare some GPP outputs (can only include screenshot here):

I have some suggestions for point 14 in the tutorial:

  1. This should start the Rose/Cylc GUI, and it will run a spin up for 35 years of 1979 before starting the main run from 1979-2012. CEDA JASMIN limits the running to 48 hours of CPU time which might only cover 20-30 years of spinup or so, so after 48 hours or so, if all went well, you can log back in to cylc1, and restart from the last dump file.
    Restarting spinup: You can restart this by editing ~/roses/u-as052/app/jules/rose-app.conf for the [namelist:jules_initial] and [namelist:jules_spinup] sections so that the idealized state lines are commented out and the ‘start from dump file’ are uncommented. You would need to change the dump file name in jules_initial to correspond to the last dump file in your output_dir: ‘file=‘/work/scratch-pw2/ndouglas/config/outputs/Euro44_bvv_nancil_CTL-BCJ-NH_jules-vn4.9p_u-as052globeA.dump.spin28.19790101.0.nc’, for example, if 28 spinup cycle have been completed. Make sure that max_spinup_cycles is decremented to correspond to the remaining number of spinup cycles that are desired, in this case set max_spinup_cycles=7. (You might want to move the old spinup files to another place beforehand, as they will be overwritten by new spinup files. Do this with the command ‘cp -R /work/scratch-pw2/Your_Username/config/outputs /work/scratch-pw2/Your_Username/config/outputs_0’, for example. The instructions for this step are particularly important if you are increasing the number of spinup years from 35 to 100.)
    Restarting main run: When max_spinup_cycles is met, the main run will start and continue until CPU time limit. To restart the main run
    make sure max_spinup_cycles is set to 0 and adjust the dump file to the last main run file. You will also have to adjust [namelist:jules_time] to correspond to the new starting time of the run. (You might want to move the spin-up files to the ‘outputs_0’ file and remove them from the ‘outputs’ so that the spin up files and the main run files will be separate after the main runs are complete. This will require renaming, use ‘cp …/…dump.spin29.19790101.0.nc …/…dump.spin29.19790101.0.nc’, for example, for both the 6 hourly and dump files.)
    I haven’t tested the ‘restart’ command-line option in Rose/Cylc for this suite.

I also have a question regarding restarting the main run after spin-up is complete - with max_spin_up_cycles=0 and we use the 2004 dump file in jules_initial, should we be using start time as 2004? Does that make sense? I seemed to have a problem once the run got to 2012 but fine otherwise.

Thanks!
Natalie

Hi Natalie
That’s great news! And thanks for the suggestion for step#14. I’ll try to update the tutorial at some point.The plots look great. What went wrong in 2012? The driving data had some missing files until recently, but then I updated the suite replacing the missing files.
Patrick

Hi again, Natalie:
What you said about changing the start time to 2004 for a 2004 restart from a dump file seems right.
Patrick

Hi Patrick,
That could be the problem, I had output for all years except 2012 and the FATAL ERROR was ‘file_ts_advance: Cannot advance to 2012-12-31 21:00:00 - data end is 2012-12-31 21:00:00’. I suspected maybe that the leap year setting could have impacted the final year but I will try running the suite again just for 2012 to see if the updated driving data rectifies the issue. To do this, I assume I would need to set ‘output_start=‘2012-01-01 00:00:00’, ‘main_run_start=‘2012-01-01 00:00:00’ and
file=’/work/scratch-pw2/ndouglas/config/outputs/Euro44_bvv_nancil_CTL-BCJ-NH_jules-vn4.9p_u-as052globeA.dump.20120101.0.nc’ for the dump file in the jules_init namelist- is that right? This last setting confuses me a little - what does a ‘dump’ actually provide?

On another note, I also got ERRORs for requesting output for times before the start of the main run that seemed to be related to the spin up in the output profile. I assume that setting output_spinup=.false. for restarting the main runs will rectify this. I will let you know how it goes!
Natalie

Hi Natalie:
If it’s a suite in the style of u-bb316, you’ll also need to change RUNSTART=1860,1,1,0,0,0
to RUNSTART=2012,1,1,0,0,0.

But I guess you’re running a suite in the style of u-as052 instead. You can open up the dump file to see what variables it contains to start that year. I didn’t actually update the driving data for u-as052, but I instead updated the driving data for u-bb316. Some of the data files for u-bb316 were missing.
Patrick