Hi CMS team,
My run of u-dq440 is failing in the atmos_main stage with a segmentation fault (core dumped):
srun: error: nid005245: tasks 320-383: Segmentation fault
srun: launch/slurm: _step_signal: Terminating StepId=11548582.0
srun: error: nid005244: tasks 256-305,307-315,318-319: Segmentation fault
srun: error: nid005248: tasks 385-389,391-394,396-409,411,413-417,420-422,424,426-432,435-440,442-446: Segmentation fault
srun: error: nid005225: tasks 0-5,7,10-14,16,20-26,28-33,35-38,40,42-52,54-63: Segmentation fault
slurmstepd: error: *** STEP 11548582.0 ON nid005225 CANCELLED AT 2025-11-14T13:34:48 ***
etc.
I can see this is happening before UKCA_TRACERS_COPY_TO_UM module, somewhere just before the command to write “Copying tracers out.”. Possibly around the ukca_step point. I have run with extra diagnostic messages. The job was running fine yesterday, however the difference is that I have set l_ukca_qch4inter = .false. and have made some edits to the code to allow this to proceed with out errors flagging up: https://code.metoffice.gov.uk/trac/um/changeset?reponame=&new=131827%40main%2Fbranches%2Fdev%2Fhannahbryant%2Fvn12.0_hyway_h2_emissions%2Fsrc%2Fatmosphere%2FUKCA&old=131791%40main%2Fbranches%2Fdev%2Fhannahbryant%2Fvn12.0_hyway_h2_emissions%2Fsrc%2Fatmosphere%2FUKCA
I am confused by this error and wondered if it might be a memory issue etc. Have you seen it before? I have 12Gb on PUMA but have only used 11Gb and have space on Archer2.
Thanks,
Hannah