Jobs not launching, constraint=ivybridge128G

Hi Martin:
Did you get it working yet?
Patrick

Hi Patrick,

I didn’t get a chance to try again this week, working on this big grant for August. I will try and have another crack next week once I have more words on the page and feel I can multi-task again. I appreciate you following up, be assured I will be following this up.

Martin

Hi Patrick,

OK running now, I will update soon.

M

Hi,

It failed at the plotting stage again. I’ve checked and all the sites worked fine so I’ve triggered the plotting script again, it has just been submitted.

M

It looks like you’re using a different version of cylc than I am.

Your job script for make_plots:
~mdekauwe/cylc-run/u-co635v2/log/job/1/make_plots/02/job
has:
export CYLC_DIR='/apps/jasmin/metomi/cylc-7.8.7'

whereas a recent version that I think worked for me:
~/cylc-run/u-al752currentworking8/log/job/1/make_plots/01/job
has:
export CYLC_DIR='/apps/contrib/metomi/cylc-7.8.1'

Maybe you can fix this by changing your .bashrc file so that it has:
export PATH=/apps/contrib/metomi/bin:$PATH
instead of:
export PATH=$PATH:/apps/contrib/metomi/bin

then:

  1. log out of cylc1 and log back in
  2. cd ~/roses/u-co635v2
  3. rose suite-run --reload. (or maybe you need to do rose suite-run --restart)
  4. rose sgc
  5. retrigger your make_plots app, by right clicking on it and choosing the retrigger option
  6. grep CYLC_DIR= ~/cylc-run/u-co635v2/log/job/1/make_plots/03/job (or whatever version you are using instead of 03)
  7. the grep should maybe say (if it doesn’t, then maybe something didn’t work as planned):
    export CYLC_DIR='/apps/contrib/metomi/cylc-7.8.1'
  8. maybe it will run properly then. Please let me know.
  9. if it doesn’t work, I might then suggest starting the suite new again, either with rose suite-run --new or by making/running a new copy of the suite with rosie or maybe just with a linux command-line copy.

Patrick

Hi Martin:
Is it working better now?
Patrick

Hi Patrick,

I just checked and no luck, it failed at the plotting step again. So, that should rule out differences in the cycl, as I’ve matched that step and I started a brand new run.

Thanks,

Martin

Hi Martin:
Maybe you can try putting some print statements especially at the beginning of the file:
/home/users/mdekauwe/cylc-run/u-co635v2/bin/make_plots.py
in order to ensure that the Python interpreter is actually properly getting into this file and then crashing?

After you put the print statements in there, you can retrigger the make_plots app in the cylc GUI. I don’t think that you need to do a rose suite-run --reload. The output of the print statements will be in the job.out log file:
/home/users/mdekauwe/cylc-run/u-co635v2/log/job/1/make_plots/NN/job.out
I think the NN version of the job.out file is the latest version.
Patrick

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Hi Martin
Did you ever get this working?
Patrick

Hi Martin
I noticed an issue with make_plots for the u-al752 suite on JASMIN. Since your ticket has been closed, I just reopened the ticket, to respond properly.
The make_plots app fails for some weird reason unless this line is deleted in the [[PLOTTING_CEDA_JASMIN]] section of site/suite.rc.CEDA_JASMIN :
env | grep LD_LIBRARY_PATH
Does that help you?
Patrick

Hi Patrick
Will test and let you know.
Martin

Hi Patrick,

I can confirm that it the whole suite ran for me when that line was removed.

Thanks,
Martin

Hi Martin:
I am very glad to hear that it works now!
The jinja language used in Rose/Cylc must be doing something that I don’t currently understand when the env | grep LD_LIBRARY_PATH is executed for an empty LD_LIBRARY_PATH path. I think that this line of code is not absolutely necessary, so I am deleting it now from the code for the u-al752 suite. The checked-in MOSRS version of this suite now doesn’t have this line of code.
There is a similar line of code elsewhere in the suite’s environment settings, where LD_LIBRARY_PATH contains actual directory names, and that seems to work fine.
Patrick