Hello,
I have been trying to follow the following link for running JULES: https://research.reading.ac.uk/landsurfaceprocesses/software-examples/tutorial-rose-cylc-jules-on-jasmin/
When reached step no. 13 and tried to run the suite the second time FCM_Make failed and showed the following attached error.
Kindly help me out as I am unable to identify the error and solve it.
Thank you
Richa
June 22:
Hi Richa
Did you look at the stderr and stdout log files for fcm_make, as described in the next step of the JULES FLUXNET tutorial (step no. 14)? What do the log files say? There are also other files in the same log directory which might be useful to look at, if the stderr and stdout logs aren’t sufficient.
Patrick McGuire
June 22:
First time i tried to compile the suite it worked fine. But due to some internet issues i was not able to get the output as the compilation was not completed.
But the second time when i tried to compile the same suite it showed FCM failed and i have no idea how to resolve it.
If anyone has faced this issue please let me know.
Richa
June 22:
I don’t have access to sci1 hence i login into cylc account and access the suite. What is the solution to the problem?
Richa
June 22:
Hi Richa
You should definitely use cylc1 instead of sci1.
Patrick
June 22:
Hi Richa
Can you try to use vi or cat or emacs or whatever, to look at the log files in the directory mentioned in step 14 of the tutorial?
Patrick
June 22:
Hi Patrick,
I’m not running the tutorial (my own suite), but I get the following (which is what Richa is apparently getting when running the tutorial)
" [FAIL] preprocess.prop{file-ext.h} = .inc: bad name-space
[FAIL] preprocess.prop{fpp.defs} = SCMA BL_DIAG_HACK INTEL_FORTRAN: bad name-space"
Jonathan
June 22:
Hi Jonathan (and Richa):
Yes, I saw the email from you and responded about it already.
What is your suite number on JASMIN? If your permissions will allow me, then I can look at your log files, to see more details beyond these two ‘[FAIL]’ lines.
That’s what I was asking you and Richa to do for yourselves in my prior emails.
The message that Richa was showing from her log file from the GUI is not very useful. It is often better to look at the log files on the command line instead of in the GUI.
The ‘[FAIL]’ lines that you quote above on the surface suggest that it might possibly have something to do with the type of processor being used.
You can also compare your suite with my suite with this command:
diff -r ~pmcguire/roses/u-al752 ~/roses/u-al752
I just restarted my suite, and it passes the fcm_make step just fine.
Patrick
June 22
Hi Patrick
The suite I am working on is u-al752. I shall see once the difference with the command as you have given in the following mails. Let me check.
Richa
June 23
Hello all,
The problem of fcm-make failure is yet unsolved and I kindly request you all to help me regarding this. Should I use some other suite rather than the trunk version of u-al752?
Error:
[FAIL] preprocess.prop{file-ext.h} = .inc: bad name-space
[FAIL] preprocess.prop{fpp.defs} = SCMA BL_DIAG_HACK INTEL_FORTRAN: bad name-space"
Or should I copy the suite first and then try the tutorial (Tutorial for setting up Rose/Cylc in order to run JULES on CEDA JASMIN - Land Surface Processes Group (reading.ac.uk))?
It would be very helpful for me if I can get some suggestions as I hope others might have faced same problems when they started.
Thanks in advance.
Richa
June 23
Hi Richa
I have been trying to help out.
What do your log files look like? I can’t see them myself.
Have you tried a ‘rose suite-run --new’?
Patrick McGuire
June 24
Hi Patrick
I was able to solve the issue of fcm_make failure by making a copy of the suite and making appropriate changes in the conf files as shown in the tutorials but the make plots step failed and displayed submission failure.
How to solve this?
I tried restarting the suite and stopping and starting it again gives me this error
file:bin/parallelise.py=source=fcm:jules.x_br/pkg/karinawilliams/r6715_python_packages/share/parallelise.py@19283: bad or missing value
Please let me know how should I proceed further.
Thank you
Richa
June 24
Hi Richa,
Following advice from Patrick, try running ‘rose suite-run --new’ That should ‘reset’ the suite and should hopefully allow you to run it.
Jon
June 24
Hi Richa:
Maybe the cylc1 node forgot your MOSRS password after a while, or something like that?
Can you try to log out of cylc1, log back in, and re-enter your password for MOSRS, and then try to retrigger the make plots app?
Patrick
June 25
Hi Richa:
Did that help?
Patrick
June 28
Hi Richa
Is it working now?
Patrick
June 29
Hi Patrick,
Sorry for the late reply.
Please tell me how to retrigger just the make plots run?
Do I need to run the complete suite run again?
Please let me know.
Thank you for helping me.
Richa
June 29
Hi Richa
You can retrigger an app by opening the suite with the rose/cylc GUI with “rose sgc”. The you can retrigger by right mouse clicking on the app and press the trigger option in the popup menu.
If you make any changes to the suite, you should first do a
‘rose suite-run --reload’
prior to retriggering the app.
Patrick
July 19
Hi Richa
Is it working now?
Patrick McGuire
July 24
Hi Patrick,
It worked when I retriggered it the third time but still it gets stuck at the make plots section and shows the error below.
ERROR: file not found: /home/users/richaiit/cylc-run/u-cf788/log/job/1/make_plots/NN/job.err
Traceback (most recent call last):
File “/apps/jasmin/metomi/cylc-7.8.7/bin/cylc-cat-log”, line 439, in
main()
File “/apps/jasmin/metomi/cylc-7.8.7/bin/cylc-cat-log”, line 435, in main
tmpfile_edit(out, options.geditor)
File “/apps/jasmin/metomi/cylc-7.8.7/bin/cylc-cat-log”, line 265, in tmpfile_edit
modtime1 = os.stat(tmpfile).st_mtime
TypeError: coercing to Unicode: need string or buffer, int found
Please tell me that as the JULES run is completed hence can I access the JULES run output files from my directory? As make plots is not working and even if it worked I would need to make my own plots.
Please let me know.
Thank you
Richa
July 26
Hi Richa
What does the make_plots job.out file say?
You should view these .out and .err files in the linux command prompt instead of from the GUI.
For example, you can use ‘vi’ or ‘more’ to look at those files at the linux command prompt.
Just use ‘ls’ to find the .out or .err file in the subdirectories of:
/home/users/richaiit/cylc-run/u-cf788/log/job/1/make_plots/
And then use ‘vi’ or ‘more’ to look at those files from the command prompt.
The GUI menu options are useful sometimes, but if you really want to understand things like
‘ERROR: file not found’, then you can look in more detail at the linux command prompt
The Traceback (most recent call last): part of the error message is not always useful with the GUI.
I can’t see your log files. Can you make the permissions so that I can see your cylc-run and roses directories? This might mean that you have to give me permission to see your whole home directory, which might mean that if you have any private or confidential information there, that you should not give permission for others to see that info.
Yes, you can access the JULES output NETCDF files in the output directory that you defined.
When you retrigger the make_plots app, you could possibly just retrigger that app, and not the whole suite.
Patrick McGuire
Aug 2
Hello Patrick,
Thanks for being considerate and helping me in this process. So here’s what I did.
I tried earlier with copying the suite and making my own copy to run the suite as directed in the tutorial, but it kept on failing and when I tried to look through the job.out file as you said then those files seemed to be empty and displayed nothing on the linux based window.
Hence, I deleted the previous checked out copy of u-al752 and downloaded it again. This time I made changes in the fcm-make.cfg file as mentioned in the given link (found through search).
#3401 (issue with JULES with suite.rc.CEDA-JASMIN) – NCAS Computational Modelling Services
Some of the changes mentioned here were already incorporated in the revised version I guess. I am not sure if there was any version conflict but I downloaded both Vn6.0 and Vn6.1.
The suite ran successfully completing each step and I could see the pdf plots after downloading them.
But I have two queries now:
module load jaspy
cd $JULES_ROOT
fcm make -j 2 -f etc/fcm-make/make.cfg --new
When I tried to use cd ~/work/scratch-nopw it shows no such directory exists in my username. I have granted the permission to look through my files for now if you can have a look at it and let me know this.
Thank you
Richa
Aug 2
Hi Richa
Today and tomorrow are official days off for my University. I will try to respond to you more when I get back on Wednesday UK time.
The output file shouldn’t be in ~/work/scratch-nopw . They are in /work/scratch-nopw .
Can we create a CMS Helpdesk ticket for all of this?
Patrick
Aug 2
Yes sure.
Please take your time.
I will try to access /work/scratch-nopw.
Thanks
Richa
Aug 4
Hi Richa
Did using /work/scratch-nopw instead of ~/work/scratch-nopw help?
Patrick
Aug 9
Hi Patrick,
Yes it worked. I was able to download files in both pdf and netcdf format from the sample run.
I shall create the ticket as per your suggestion.
Also, now I am looking to feed my own data into the suite and run it for my own study area. Please guide me if you have suggestions for it. I am planning to use u-al752 and modify it according to my needs.
Kindly suggest how I should start doing that.
I have a suggestion which can help new users to begin with JULES and reduce their struggle. The tutorial on the given (Tutorial for setting up Rose/Cylc in order to run JULES on CEDA JASMIN - Land Surface Processes Group (reading.ac.uk)) page is very descriptive and it has helped a lot. But it will be more helpful if there is a video tutorial for the same. Starting from the downloading of the suite to displaying the output. That link could be added on the same page in the end for users to get familiar with it. I don’t know if this has been thought of. The video tutorials from Toby helped me a lot and I thought why not for this page.
Thank you
Richa
Aug 9
Hi Richa
I am glad you can download the PDF and NETCDF files now from the JULES FLUXNET suite. How do the plots look?
Yes, please feel free to modify the u-al752 suite for your purposes. Do you know how to make a Rose/Cylc copy of the suite? Then you can have editing privileges. Do you know how to check in your changes to the suite into the MOSRS? Also, have a look at the Fair Use policy on the TRAC site for the JULES FLUXNET suite u-al752 https://code.metoffice.gov.uk/trac/jules/wiki/FluxnetandLbaSites.
I am glad that the video tutorials from Toby helped you a lot. That’s what they are there for. I don’t have enough resources (i.e., time) right now to make video tutorials for the JULES FLUXNET suite installation, but thanks for the suggestion.
Please respond to this ticket either by replying to the email, or by going to the CMS Helpdesk website and responding there. I am limited to 3 responses before someone else has to respond, and I have used up all 3 of my responses filling in the history of this ticket on the new Helpdesk.
Are you working with anyone on this project, either in the UK or in India? What kind of data do you want to put in to your version of the JULES FLUXNET suite? Maybe you can start a new ticket to the CMS Helpdesk on this? Or feel free to continue this ticket if you want.
Patrick
Thanks for your email. The plots look good and similar to ones in the pdf plots.
I am still not clear with the part that how to feed my own data (i.e. forcing data, ancillary data) into the model for it to run. Well I posted the same in users community and I got a reply to use create_rose_app for this work. But it is still not clear. I have a few questions:
Please let me know the answers to these questions.
I want to use my own data and not the FLUXNET data. Is this possible by using the u-al752 suite ?
If not, what is the process to run the model with our own data?
Thanks
Richa
Hi Richa:
As far as the format for the forcing data and the ancillary data, you can look at the format of the forcing data and the ancillary data that is currently used by the suite.
You’ll see the SUITE_DATA variable defined in the rose-suite.conf file. You can look at the files in the path defined by the SUITE_DATA variable. You’ll also see the DATA_VERSION defined in the suite.rc file, which will help you to find the right subdirectory of the SUITE_DATA path.
You’ll also see the app/jules/opt files for each site. You can look at the contents of those files and the paths of the ancillary files that those files point to, to see the format of the different data files that we use.
You can then try to make your driving data and ancillary files to be of the same format. You’ll need to write your own code to make those files.
Patrick