# Grouped Collapses in cf-python

**URL:** <https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593>\
**Category:** General\
**Tags:** cfpython\
**Created:** [13 December 2024 21:40 UTC](https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593 "2024-12-13T21:40:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ajw1g19](https://avatars.discourse-cdn.com/v4/letter/a/35a633/32.png) [@ajw1g19](https://cms-helpdesk.ncas.ac.uk/u/ajw1g19)\
**Post date:** [13 December 2024 21:40 UTC](https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593/1 "2024-12-13T21:40:39Z")

</div>

Dear CMS Helpdesk,

I’m attempting to use cf-python to evaluate monthly means for hourly climate diagnositcs. The diagnostics are themselves monthly means, output for particular hours in the day (00:00, 03:00, 06:00, etc) using the time domain TMONMNXY where XY is the hour-code. So for every year there are 96 time slices, 8 for each month.

I am attempting to collapse 80 years worth of these diagnostics into a single set of monthly means where each time represents the average of that time over all 80 years.

I’ve tried using the following to perform this operation:

```auto
field.collapse("T: mean within years T: mean over years",within_years = cf.month(mon) & cf.hour(hrs[idx]) & cf.minute(mins[idx]))

```

This sits in a loop that iterates over integers for each month and then over integers for each combination of the hours and minutes that form the hourly time slices.

The code appears to work for the first iteration (January means at 00:00), where:

```auto
field.collapse("T: mean within years T: mean over years",within_years = cf.month(1) & cf.hour(0) & cf.minute(0))

```

but for the next iteration (selecting the January means at 03:00), it fails and produces the following error:

```auto
field.collapse("T: mean within years T: mean over years",within_years = cf.month(1) & cf.hour(1) & cf.minute(30))
ValueError: Can't collapse: No contiguous groups within years were identified

```

Can you explain what’s going wrong?  
Also, if this is a question that should be posted elsewhere, please do let me know.

Regards,  
Alfred

---

<div class="post-metadata">

**Author:** ![sadielbartholomew](https://dub1.discourse-cdn.com/flex013/user_avatar/cms-helpdesk.ncas.ac.uk/sadielbartholomew/32/436_2.png) [@sadielbartholomew](https://cms-helpdesk.ncas.ac.uk/u/sadielbartholomew)\
**Post date:** [17 December 2024 16:43 UTC](https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593/2 "2024-12-17T16:43:57Z")

</div>

Hi Alfred,

Thanks for providing all the detail on this.

Firstly, there may be cases where the groups are not contiguous (e.g. where there are missing required time data or overlapping time bounds) in which case you will need to set `group_contiguous=0` in your call to prevent failures from that. That may immediately work. If not, if I understand your desired result correctly, the issue could be the format in which you are specifying your `within_years` groups. You have given these as a query which evaluates to, for example in the failing case you state:

```auto
>>> cf.month(1) & cf.hour(1) & cf.minute(30)
<CF Query: [[month(eq 1) & hour(eq 1)] & minute(eq 30)]>

```

but what we actually need to specify for a monthly mean is to specify a time _duration_ of a month for a specific start time e.g. your hour at 1 and minute at 30, for example, like so `cf.M(1, hour=1, minute=30)`.

So, if you try this (in combination with the above parameter setting, to check), it may then work:

```auto
f.collapse("T: mean within years", within_years=cf.M(mon, hour=hrs[idx], minute=mins[idx]), group_contiguous=0)

```

Please let me know if either of those are helpful and if they aren’t we can investigate further.

Thanks,

Sadie

---

<div class="post-metadata">

**Author:** ![ajw1g19](https://avatars.discourse-cdn.com/v4/letter/a/35a633/32.png) [@ajw1g19](https://cms-helpdesk.ncas.ac.uk/u/ajw1g19)\
**Post date:** [19 December 2024 12:38 UTC](https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593/3 "2024-12-19T12:38:38Z")

</div>

Hi Sadie,

Thanks for the advice, I tried what you suggested but unfortunately got the following error for time slices at January 03:00:

```auto
ValueError: Can't collapse: No groups within years spanning P1M (Y-M-01 01:30:00) were identified

```

Luckily I found that I can use indexing to subset the field for the correct time slices in each iteration of the loop:

```auto
for sample in range(nsamples):
     slice = np.arange(sample,field.shape[0],nsamples)
     mon_mns.append(field[slice,:].collapse("T: mean",coordinate="minimum"))

```

`nsamples` is the number of samples in one year (96) and `slice` is a list of all indices in the time domain that correspond to the same monthly and hourly slice.  
The subset field is then meaned using `cf.collapse` and the result appended to a list (`mon_mns`).  
Finally, the `nsamples` meaned fields in `mon_mns` are combined with `cf.aggregate` into a single construct to be written to netcdf.

This approach actually works out much faster than trying to use `"T: mean within years T: mean over years"`

Thanks again for your help!

Regards,  
Alfred

---

<div class="post-metadata">

**Author:** ![system](https://europe1.discourse-cdn.com/flex013/uploads/cms_support/original/1X/1fd2411499ffcbc299fe756cd5cdf26e44956558.png) [@system](https://cms-helpdesk.ncas.ac.uk/u/system)\
**Post date:** [18 January 2025 12:39 UTC](https://cms-helpdesk.ncas.ac.uk/t/grouped-collapses-in-cf-python/1593/4 "2025-01-18T12:39:16Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.