Hour of day categorised coord does not return correct hour coord for aggregated cube

Peter Willetts

unread,

May 21, 2015, 7:00:01 AM5/21/15

to scitoo...@googlegroups.com

Hi

I am trying to aggregate mean by hour of day. Which it would appear is happening, but the 'hour' coord on the returned mean cube has repeating values for 1 to 6 (i.e no 13-18), which are in the original cube.

This is using version 1.8.0

Thanks

Peter

import iris
import iris.unit as unit
from iris.coord_categorisation import add_categorised_coord

def add_hour_of_day(cube, coord, name='hour'):
    add_categorised_coord(cube, name, coord,
          lambda coord, x: coord.units.num2date(x).hour)


time_coords = cube.coord('time')
add_hour_of_day(cube, time_coords)

mean = cube.aggregated_by('hour', iris.analysis.MEAN)

mean.coord('hour')

AuxCoord(array([12, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,

11, 1, 2, 3, 4, 5, 6]), standard_name=None, units=Unit('1'), long_name=u'hour')

cube.coord('hour')

AuxCoord(array([ 0, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,

23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,

16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8,

9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1,

2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,

19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,

12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4,

5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,

22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,

15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7,

8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 1, 2, 3, 4,

5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,

22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,

15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 7, 8, 9, 10, 11, 12, 13,

14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6,

7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,

17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,

10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 1, 2, 3,

4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,

21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,

14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 7, 8, 9, 10,

11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3,

4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,

21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,

14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6,

7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]), standard_name=None, units=Unit('1'), long_name=u'hour')

print mean

y_wind / (m s-1) (-- : 24; pressure: 12; latitude: 187; longitude: 140)

Dimension coordinates:

pressure - x - -

latitude - - x -

longitude - - - x

Auxiliary coordinates:

hour x - - -

time x - - -

Attributes:

STASH: m01s30i202

source: Data from Met Office Unified Model

um_version: 8.2

Cell methods:

mean: hour

print cube

y_wind / (m s-1) (time: 459; pressure: 12; latitude: 187; longitude: 140)

Dimension coordinates:

time x - - -

pressure - x - -

latitude - - x -

longitude - - - x

Auxiliary coordinates:

hour x - - -

Attributes:

STASH: m01s30i202

source: Data from Met Office Unified Model

um_version: 8.2

Peter Willetts

unread,

May 21, 2015, 7:19:53 AM5/21/15

to scitoo...@googlegroups.com

The 'time' coord is the same

def ConvertHoursSince1970ToDatetime(hours_since):
    """ Takes numpy array of 'hours since 1970' and string format
     and returns numpy array of date strings"""
    u = unit.Unit('hours since 1970-01-01 00:00:00',calendar='gregorian')
    return np.array([u.num2date(da) for da in np.array(hours_since)])

ConvertHoursSince1970ToDatetime(mean.coord('time').points)

array([datetime.datetime(2011, 8, 28, 12, 0),

datetime.datetime(2011, 8, 28, 19, 0),

datetime.datetime(2011, 8, 28, 20, 0, 0, 13),

datetime.datetime(2011, 8, 28, 21, 0),

datetime.datetime(2011, 8, 28, 22, 0),

datetime.datetime(2011, 8, 28, 23, 0, 0, 13),

datetime.datetime(2011, 8, 29, 0, 0),

datetime.datetime(2011, 8, 29, 1, 0),

datetime.datetime(2011, 8, 29, 2, 0, 0, 13),

datetime.datetime(2011, 8, 29, 3, 0),

datetime.datetime(2011, 8, 29, 4, 0),

datetime.datetime(2011, 8, 29, 5, 0, 0, 13),

datetime.datetime(2011, 8, 29, 6, 0),

datetime.datetime(2011, 8, 29, 7, 0),

datetime.datetime(2011, 8, 29, 8, 0, 0, 13),

datetime.datetime(2011, 8, 29, 9, 0),

datetime.datetime(2011, 8, 29, 10, 0),

datetime.datetime(2011, 8, 29, 11, 0, 0, 13),

datetime.datetime(2011, 8, 29, 1, 0),

datetime.datetime(2011, 8, 29, 2, 0, 0, 13),

datetime.datetime(2011, 8, 29, 3, 0),

datetime.datetime(2011, 8, 29, 4, 0),

datetime.datetime(2011, 8, 29, 5, 0, 0, 13),

datetime.datetime(2011, 8, 29, 6, 0)], dtype=object)

Peter Willetts

unread,

May 21, 2015, 7:34:11 AM5/21/15

to scitoo...@googlegroups.com

I get the same thing doing this. It appears the time scalar coordinate is incorrect, although the bounds are.

Thanks

Peter

hours = np.unique(cube.coord('hour').points)


cubelist=[]


for h in hours:


    cubelist.append(cube[cube.coord('hour').points==h].collapsed(('time'), iris.analysis.MEAN))


cube_merge = iris.cube.CubeList(cubelist).merge_cube()

print cubelist[15]

y_wind / (m s-1) (pressure: 12; latitude: 187; longitude: 140)

Dimension coordinates:

pressure x - -

latitude - x -

longitude - - x

Scalar coordinates:

hour: 15, bound=(15, 15)

time: 2011-08-29 03:00:00, bound=(2011-08-19 15:00:00, 2011-09-07 15:00:00)

Attributes:

STASH: m01s30i202

source: Data from Met Office Unified Model

um_version: 8.2

Cell methods:

mean: time

On Thursday, 21 May 2015 12:00:01 UTC+1, Peter Willetts wrote:

Andrew Dawson

unread,

May 21, 2015, 11:27:31 AM5/21/15

to scitoo...@googlegroups.com

I tried to make a simple self-contained example from your post, but I can't reproduce the issue. As a first step can you try running the attached test program, it will print a message to the terminal if it triggers a bug, nothing otherwise.

hour_aggregation.py

Peter Willetts

unread,

May 21, 2015, 11:57:10 AM5/21/15

to scitoo...@googlegroups.com

Hi

Nothing printed.

I've found a work around now, but.

One thing I was wondering about is that the cube I have been using was 'flattened' from time being in 2 coordinates like so?

def FlattenIfTimeMultipleCoordsSave(save_dir, cube_list, diag, exp_id):

    ''' 

    Takes a base directory , a loaded iris cube list, diagnostic name (for saving cubes) 

    and experiment_id 

     
    Returns nothing, saves cubes in ..._flatten.pp 

    ''' 
     
    auxcoord='time' 

    for cube in cube_list: 


        if len(cube.coord(auxcoord).points.shape)>1: 

    
            other_dim_coords = [cube.dim_coords[i].name() for i in range(cube.ndim) if i not in cube.coord_dims(auxcoord)] 

         
            for t, time_cube in enumerate(cube.slices(other_dim_coords)): 

                time_cube.remove_coord('forecast_period') 

                time_cube.remove_coord('forecast_reference_time') 

                iris.save(time_cube, '%s%s_%s_flatten.pp' \ 

                          % (save_dir, diag.replace('.pp', ''), exp_id), append=True)

x_wind / (m s-1) (forecast_period: 6; forecast_reference_time: 79; pressure: 12; grid_latitude: 599; grid_longitude: 600)

Dimension coordinates:

forecast_period x - - - -

forecast_reference_time - x - - -

pressure - - x - -

grid_latitude - - - x -

grid_longitude - - - - x

Auxiliary coordinates:

time x x - - -

Attributes:

STASH: m01s30i201

source: Data from Met Office Unified Model

um_version: 8.2

Reply all

Reply to author

Forward