Day/night image

134 views
Skip to first unread message

Thanasis Karagiannidis

unread,
Apr 23, 2020, 4:53:28 AM4/23/20
to pyt...@googlegroups.com

Dear all

 

I wish to make an image like the following (but maybe for a smaller domain) using msg hrit data and 2 static backgrounds (one for day and one for night). I tried the DayNightCompositor but it doesn’t seem to work.

I think I do not have the correct compositors etc

Can anyone direct me on this?

 

Sakis

59088056-0e5cfa80-890f-11e9-8562-39b133d5731e.png

Panu Lahtinen

unread,
Apr 23, 2020, 4:59:47 AM4/23/20
to pyt...@googlegroups.com
How does it not work for you? Any error messages, stack traces or the
like you could show us?

The compositor it self certainly works, and in Pytroll Slack there have
been several examples on the composites you showed.


P

Thanasis Karagiannidis

unread,
Apr 23, 2020, 5:16:31 AM4/23/20
to pyt...@googlegroups.com
Dear Panu


In my script i set

from satpy.composites import DayNightCompositor
compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)
composite = compositor([global_scene['clouds_with_background_day'], global_scene['night_ir_with_background_hires']])

and when I run it I get


Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 307, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'clouds_with_background_day'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "msg4_Europe_merc_daynight.py", line 190, in <module>
    composite = compositor([global_scene['clouds_with_background_day'], global_scene['night_ir_with_background_hires']])
  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 681, in __getitem__
    return self.datasets[key]
  File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 309, in __getitem__
    key = self.get_key(item)
  File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 298, in get_key
    best=best, **dfilter)
  File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 249, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"

However when I use "clouds_with_background_day" and " night_ir_with_background_hires" on their own scripts everything works just fine and the images are created.

Sakis

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/9fa7329d-c2ba-bad0-d7ed-4560e0cca842%40fmi.fi.

David Hoese

unread,
Apr 23, 2020, 8:52:19 AM4/23/20
to pyt...@googlegroups.com
Sakis,

Could you show us the part of the code where you create and load data in
`global_scene`? The error you are getting is suggesting the
`clouds_with_background_day` composite hasn't been fully loaded and
doesn't exist. If I had to guess (without looking at the code) I would
guess that you need to resample your `global_scene` so that Satpy is
able to create the `clouds_with_background_day` composite.

Dave
> <mailto:pytroll%2Bunsu...@googlegroups.com>.
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thanasis Karagiannidis

unread,
Apr 23, 2020, 9:59:28 AM4/23/20
to pyt...@googlegroups.com

Dave I tried to load and resample the 2 global scenes. The fist one is resampled just fine but the second one crashes. Should I somehow “delete” the first global scene before moving to the next one? I include my part of the code and the outputs

 

Related part of script

 

composite_1 = 'clouds_with_background_day'

composite_2 = 'night_ir_with_background_hires' 

area = 'Europe_merc'

files = glob(tmpdir + "/" + "H-000-MSG*" + Dat + "*")

global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

 

# Loading and resampling comp 1

global_scene.load([composite_1])

resample_method = 'bilinear'

new_scene_1 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir')

print("ok scene 1")

# Loading and resampling comp 2

global_scene.load([composite_2])

resample_method = 'bilinear'

new_scene_2 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir')

print("ok scene 2")

 

from satpy.composites import DayNightCompositor

compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)

composite = compositor([new_scene_1['clouds_with_background_day'], new_scene_2['night_ir_with_background_hires']])

 

 

Output of run

 

/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

  data = yaml.load(f.read()) or {}

The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

ok scene 1

The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

Traceback (most recent call last):

  File "msg4_Europe_merc_daynight.py", line 196, in <module>

    new_scene_2 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir')

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample

    reduce_data=reduce_data, **resample_kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene

    res = resample_dataset(dataset, destination_area, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset

    new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample

    res = resampler_instance.resample(data, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample

    return self.compute(data, cache_id=cache_id, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute

    output_shape=target_shape)

  File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info

    res = DataArray(res, dims=data.dims, coords=self.out_coords)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__

    coords, dims = _infer_coords_and_dims(data.shape, coords, dims)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims

    'coordinate %r' % (d, sizes[d], s, k))

ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'

 

 

Sakis

 

 

 





To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.

David Hoese

unread,
Apr 23, 2020, 10:25:34 AM4/23/20
to pyt...@googlegroups.com
That error sounds like something is wrong with the composites, but let's
try something different before giving up. I would specify boths
composites in your first `.load` call then only have one resample. Since
you are resampling both of them in the same way there is no need to
split this work. The way you have it now you are actually resampling
`composite_1` twice. That might be what's leading to this odd error.

So your code should then be structured as:

global_scene = Scene(...)
global_scene.load([composite_1, composite_2])
new_scene = global_scene.resample(...)
# and so on

Dave

On 4/23/20 8:59 AM, Thanasis Karagiannidis wrote:
> Dave I tried to load and resample the 2 global scenes. The fist one is
> resampled just fine but the second one crashes. Should I somehow
> “delete” the first global scene before moving to the next one? I include
> my part of the code and the outputs
>
> Related part of script
>
> composite_1 = 'clouds_with_background_day'
>
> composite_2 = 'night_ir_with_background_hires'
>
> area = 'Europe_merc'
>
> files = glob(tmpdir + "/" + "H-000-MSG*" + Dat + "*")
>
> global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
>
> # Loading and resampling comp 1
>
> global_scene.load([composite_1])
>
> resample_method = 'bilinear'
>
> new_scene_1 = global_scene.resample(area, radius_of_influence=10000,
> resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir')
>
> print("ok scene 1")
>
> # Loading and resampling comp 2
>
> global_scene.load([composite_2])
>
> resample_method = 'bilinear'
>
> new_scene_2 = global_scene.resample(area, radius_of_influence=10000,
> resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir')
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir')
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>.
> >     To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/9fa7329d-c2ba-bad0-d7ed-4560e0cca842%40fmi.fi.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pytroll" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com
>
> >
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thanasis Karagiannidis

unread,
Apr 23, 2020, 11:55:22 AM4/23/20
to pyt...@googlegroups.com

I tried

 

global_scene.load([composite_1,composite_2])

 

but the error continued

 

/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

  data = yaml.load(f.read()) or {}

The following datasets were not created and may require resampling to be generated: DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

Traceback (most recent call last):

  File "msg4_Europe_merc_daynight.py", line 191, in <module>

    new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir')

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample

    reduce_data=reduce_data, **resample_kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene

    res = resample_dataset(dataset, destination_area, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset

    new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample

    res = resampler_instance.resample(data, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample

    return self.compute(data, cache_id=cache_id, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute

    output_shape=target_shape)

  File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info

    res = DataArray(res, dims=data.dims, coords=self.out_coords)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__

    coords, dims = _infer_coords_and_dims(data.shape, coords, dims)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims

    'coordinate %r' % (d, sizes[d], s, k))

ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'

 

BUT, when I changed my resampling method to nearest

 

resample_method = 'nearest'

new_scene = global_scene.resample(area, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir')

 

the script continued but crashed later, when I tried to save the dataset

 

Traceback (most recent call last):

  File "msg4_Europe_merc_daynight.py", line 275, in <module>

    new_scene.save_dataset(composite,'./MSG4.png')

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1295, in save_dataset

    return writer.save_dataset(self[dataset_id],

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 681, in __getitem__

    return self.datasets[key]

  File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 307, in __getitem__

    return super(DatasetDict, self).__getitem__(item)

TypeError: unhashable type: 'DataArray'

 

So, firstly for some reason the resampling using bilinear fails although it has no problem when used in the exact same way in separate scripts. Could it be something related to the projections or resolutions of the specific composites? To support that, when I replace these composites with “day_microphysics” and “night_microphysics” no resampling is required before I use the DayNightCompositor (I tried that). But besides that, why does the new_scene.save_dataset(composite,'./MSG4.png') fails?


Sakis


To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/b096eea7-c5bd-fbd3-85df-ee1ed4b276d9%40gmail.com.

David Hoese

unread,
Apr 23, 2020, 12:58:08 PM4/23/20
to pyt...@googlegroups.com
For bilinear resampling, I wonder if your cache directory got corrupt.
It might be worth deleting the files in there and trying again.
Otherwise we might need to pull Panu back in this conversation since he
knows more about the bilinear resampling than I do.

However, the nearest neighbor version of your code shows that you are
passing the wrong type of parameter to save_dataset. The "composite"
variable you have is the actual DataArray object returned from the
compositor. The "save_dataset" method expects the *name* of the dataset
that you want to save, not the DataArray object. You should add your new
DataArray object back to the Scene and then call save_dataset like this:

new_scn['dnc'] = composite
new_scn.save_dataset('dnc', 'MSG4.png')

Dave


On 4/23/20 10:55 AM, Thanasis Karagiannidis wrote:
> I tried
>
> /*global_scene.load([composite_1,composite_2])*/
>
> but the error continued
>
> //*usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning:
> calling yaml.load() without Loader=... is deprecated, as the default
> Loader is unsafe. Please read https://msg.pyyaml.org/load for full
> details.*/
>
> /**/
>
> /*data = yaml.load(f.read()) or {}*/
>
> /**/
>
> /*The following datasets were not created and may require resampling to
> be generated: DatasetID(name='night_ir_with_background_hires',
> wavelength=None, resolution=None, polarization=None, calibration=None,
> level=None, modifiers=None),
> DatasetID(name='clouds_with_background_day', wavelength=None,
> resolution=None, polarization=None, calibration=None, level=None,
> modifiers=None)*/
>
> /**/
>
> /*Traceback (most recent call last):*/
>
> /**/
>
> /*File "msg4_Europe_merc_daynight.py", line 191, in <module>*/
>
> /**/
>
> /*new_scene = global_scene.resample(area, radius_of_influence=10000,
> resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir')*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 1105, in resample*/
>
> /**/
>
> /*reduce_data=reduce_data, **resample_kwargs)*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 1060, in _resampled_scene*/
>
> /**/
>
> /*res = resample_dataset(dataset, destination_area, **kwargs)*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 1305, in resample_dataset*/
>
> /**/
>
> /*new_data = resample(source_area, dataset, destination_area,
> fill_value=fill_value, **kwargs)*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 1268, in resample*/
>
> /**/
>
> /*res = resampler_instance.resample(data, **kwargs)*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 418, in resample*/
>
> /**/
>
> /*return self.compute(data, cache_id=cache_id, **kwargs)*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 895, in compute*/
>
> /**/
>
> /*output_shape=target_shape)*/
>
> /**/
>
> /*File
> "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py",
> line 195, in get_sample_from_bil_info*/
>
> /**/
>
> /*res = DataArray(res, dims=data.dims, coords=self.out_coords)*/
>
> /**/
>
> /*File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line
> 229, in __init__*/
>
> /**/
>
> /*coords, dims = _infer_coords_and_dims(data.shape, coords, dims)*/
>
> /**/
>
> /*File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line
> 84, in _infer_coords_and_dims*/
>
> /**/
>
> /*'coordinate %r' % (d, sizes[d], s, k))*/
>
> /**/
>
> /*ValueError: conflicting sizes for dimension 'bands': length 4 on the
> data but length 2 on coordinate 'bands'*/
>
> BUT, when I changed my resampling method to nearest
>
> /*resample_method = 'nearest'*/
>
> /**/
>
> /*new_scene = global_scene.resample(area, resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir')*/
>
> the script continued but crashed later, when I tried to save the dataset
>
> /*Traceback (most recent call last):*/
>
> /**/
>
> /*File "msg4_Europe_merc_daynight.py", line 275, in <module>*/
>
> /**/
>
> /*new_scene.save_dataset(composite,'./MSG4.png')*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 1295, in save_dataset*/
>
> /**/
>
> /*return writer.save_dataset(self[dataset_id],*/
>
> /**/
>
> /*File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 681, in __getitem__*/
>
> /**/
>
> /*return self.datasets[key]*/
>
> /**/
>
> /*File
> "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line
> 307, in __getitem__*/
>
> /**/
>
> /*return super(DatasetDict, self).__getitem__(item)*/
>
> /**/
>
> /*TypeError: unhashable type: 'DataArray'*/
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>.
> >      >     To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/9fa7329d-c2ba-bad0-d7ed-4560e0cca842%40fmi.fi.
> >      >
> >      > --
> >      > You received this message because you are subscribed to
> the Google
> >      > Groups "pytroll" group.
> >      > To unsubscribe from this group and stop receiving emails
> from it,
> >     send
> >      > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>>.
> >      > To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com
> >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >
> >     --
> >     You received this message because you are subscribed to the
> Google
> >     Groups "pytroll" group.
> >     To unsubscribe from this group and stop receiving emails from it,
> >     send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>.
> >     To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pytroll" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com
>
> >
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/b096eea7-c5bd-fbd3-85df-ee1ed4b276d9%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thanasis Karagiannidis

unread,
Apr 23, 2020, 1:53:03 PM4/23/20
to pyt...@googlegroups.com

When I added

new_scene['dnc'] = composite

new_scene.save_dataset('dnc', 'MSG4.png')

 

the image was saved. But something is still wrong. With fill_value=0 in resampling (nearest) I got the image FV0.jpg while without fill_value=0 sapty set 255 for fill values (“Fill value incompatible with integer data using 255 instead.”) and I got image FV255.jpg. As for the clearing of the cache directory I tried it but it did not helped.

 

Sakis


To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/f1e63fe6-818b-d120-5ef8-a1363a9e1e16%40gmail.com.
FV255.jpg
FV0.jpg

David Hoese

unread,
Apr 23, 2020, 3:46:05 PM4/23/20
to pyt...@googlegroups.com
There are two things at play here:

1. Typically you don't need to specify `fill_value` to the "resample"
call. It assumes that NaN represents missing values and it will use NaN
on the output resampled data. There are special cases builtin for
integer data (ex. category products) where
`some_data_arr.attrs['_FillValue']` is used instead of NaN. If no
FillValue is found and none is specified then resampling will default to
the maximum for the data type (255 for unsigned 8-bit integers). I'm not
familiar with your "clouds_with_background_day" composite. Is that
builtin to Satpy? Did you make it yourself?

2. During the saving process Satpy performs "enhancements" on the data
to scale it. For your composite you probably want this hardcoded to
linearly scale the data from a 0-1 range to 0-255 to produce an 8-bit
image. If you composite doesn't have a enhancement configured then Satpy
will default to linearly stretching it from the min/max of the data.
This can sometimes produce odd results. Here are some examples that are
configured in Satpy right now:

https://github.com/pytroll/satpy/blob/master/satpy/etc/enhancements/generic.yaml#L816-L834

It is possible one of these composites is misconfigured in Satpy or in
your own configuration. I'm guessing neither of the images you sent
before were what you wanted?

Dave
> >      >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>
> >     <mailto:pytroll%25252Bun...@googlegroups.com
> <mailto:pytroll%2525252Bu...@googlegroups.com>>>>.
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com
> >      >
> >      >      >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >      >
> >      >     --
> >      >     You received this message because you are subscribed
> to the
> >     Google
> >      >     Groups "pytroll" group.
> >      >     To unsubscribe from this group and stop receiving
> emails from it,
> >      >     send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>.
> >      >     To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.
> >      >
> >      > --
> >      > You received this message because you are subscribed to
> the Google
> >      > Groups "pytroll" group.
> >      > To unsubscribe from this group and stop receiving emails
> from it,
> >     send
> >      > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>>.
> >      > To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com
> >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >
> >     --
> >     You received this message because you are subscribed to the
> Google
> >     Groups "pytroll" group.
> >     To unsubscribe from this group and stop receiving emails from it,
> >     send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>.
> >     To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/b096eea7-c5bd-fbd3-85df-ee1ed4b276d9%40gmail.com.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pytroll" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com
>
> >
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/f1e63fe6-818b-d120-5ef8-a1363a9e1e16%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z7BCWsoYkJnotFS8Sk1dxP0nJRcJ%3DZEUO4pqqhym-NCA%40mail.gmail.com
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z7BCWsoYkJnotFS8Sk1dxP0nJRcJ%3DZEUO4pqqhym-NCA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thanasis Karagiannidis

unread,
Apr 24, 2020, 4:46:05 AM4/24/20
to pyt...@googlegroups.com

Ok, a lot of issues! To summarize:

 

1.        The “clouds_with_background_day” was created following the example of Christian Peters (mail entitled “colorize image”) with the following

seviri.yaml

clouds_with_background_day:

    compositor: !!python/name:satpy.composites.BackgroundCompositor

    standard_name: clouds_with_background_day

    prerequisites:

      - ir_cloud_day

      - static_day

static_day:

    compositor: !!python/name:satpy.composites.StaticImageCompositor

    standard_name: static_night

    filename: /xyz/satpy/backgrounds/world.200407.3x21600x10800.png

    area: EPSG_4326

 

generic.yaml

static_day:

    standard_name: static_day

    operations:

    - name: stretch

      method: *stretchfun

      kwargs:

        stretch: crude

        min_stretch: [0, 0, 0]

        max_stretch: [255, 255, 255]

 

areas.yaml

EPSG_4326:

   description: Global equal latitude/longitude grid

   projection:

     init: EPSG:4326

   shape:

     width: 21600

     height: 10800

   area_extent:

     lower_left_xy: [-180.0, -90.0]

     upper_right_xy: [180.0, 90.0]

 

while I downloaded and put in a directory the

https://neo.sci.gsfc.nasa.gov/archive/bluemarble/bmng/world_2km/world.200407.3x21600x10800.png

 

2.        The enhancements are configured exactly in the way you proposed.

 

No, the images are not was I wanted. I need to get a “mix” of the two images like those I sent here (possibly for a European domain – I can change that with no problem) that look like the one I sent in my first e-mail and reattach now.  As you can see, my scripts work with no problem when I use the two composites ('clouds_with_background_day' and 'night_ir_with_background_hires') separately.

Up to now the related lines of my script are shown below. If anyone could help I would be grateful. I think we are very close to succeed!

 

Parts of script

composite_1 = 'clouds_with_background_day'

composite_2 = 'night_ir_with_background_hires'         

area = 'Europe_merc'

area_def = get_area_def(area)

global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

global_scene.load([composite_1,composite_2])

#resample_method = 'nearest'

#new_scene = global_scene.resample(area, resampler = resample_method, cache_dir='/home/thankar/Pytroll/cache_dir_2')

resample_method = 'bilinear'

new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir_2')

from satpy.composites import DayNightCompositor

compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)

composite = compositor([new_scene['clouds_with_background_day'], new_scene['night_ir_with_background_hires']])

new_scene['dnc'] = composite

new_scene.save_dataset('dnc', 'MSG4.png')

 

when I use the “nearest” resampler the script gives the following output and produces the incorrect day-night image I sent earlier.

 

/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

  data = yaml.load(f.read()) or {}

The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

Fill value incompatible with integer data using 255 instead.

Fill value incompatible with integer data using 255 instead.

 

 

When I use the “bilinear” resampler the script crashes with the following error.

 

/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

  data = yaml.load(f.read()) or {}

The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

Traceback (most recent call last):

  File "msg4_Europe_merc_daynight.py", line 197, in <module>

    new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0,  cache_dir='/home/thankar/Pytroll/cache_dir_2')

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample

    reduce_data=reduce_data, **resample_kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene

    res = resample_dataset(dataset, destination_area, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset

    new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample

    res = resampler_instance.resample(data, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample

    return self.compute(data, cache_id=cache_id, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute

    output_shape=target_shape)

  File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info

    res = DataArray(res, dims=data.dims, coords=self.out_coords)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__

    coords, dims = _infer_coords_and_dims(data.shape, coords, dims)

  File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims

    'coordinate %r' % (d, sizes[d], s, k))

ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'


Sakis

To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/ea50203f-76d8-7f2d-9ac2-d785550ffe7d%40gmail.com.
59088056-0e5cfa80-890f-11e9-8562-39b133d5731e.png
night_clouds.jpg
day_clouds.jpg

David Hoese

unread,
Apr 24, 2020, 10:03:23 AM4/24/20
to pyt...@googlegroups.com
1. First thing I notice is that the `standard_name` for the composite is
"static_night" and the name of the composite is "static_day". That's not
necessarily a problem, but the enhancement has "standard_name:
static_day" which means it onlly applies to composites with that
standard_name. This should be switched to static_night (or copy/paste so
you have one section for static_day, one for static_night). It overall
probably isn't a huge deal, but could still be a problem in the future.

2. I'm not so sure what's going on here. Are you sure that code is
giving those warnings? You are specifying the fill_value=0 and still
getting the warning about Fill value incompatible? We might have to
check with Christian on if he's seen this kind of stuff with these
composites in the past. As for the bilinear exception, I've asked Panu
to take a look but he's taking a couple days off so it might be a while.

Dave

On 4/24/20 3:45 AM, Thanasis Karagiannidis wrote:
> Ok, a lot of issues! To summarize:
>
> 1.The “clouds_with_background_day” was created following the example of
> Christian Peters (mail entitled “colorize image”) with the following
>
> */seviri.yaml/*
>
> /clouds_with_background_day:/
>
> /compositor: !!python/name:satpy.composites.BackgroundCompositor/
>
> /standard_name: clouds_with_background_day/
>
> /prerequisites:/
>
> /- ir_cloud_day/
>
> /- static_day/
>
> /static_day:/
>
> /compositor: !!python/name:satpy.composites.StaticImageCompositor/
>
> /standard_name: static_night/
>
> /filename: /xyz/satpy/backgrounds/world.200407.3x21600x10800.png/
>
> /area: EPSG_4326/
>
> */generic.yaml/*
>
> /static_day:/
>
> /standard_name: static_day/
>
> /operations:/
>
> /- name: stretch/
>
> /method: *stretchfun/
>
> /kwargs:/
>
> /stretch: crude/
>
> /min_stretch: [0, 0, 0]/
>
> /max_stretch: [255, 255, 255]/
>
> */areas.yaml/*
>
> /EPSG_4326:/
>
> /description: Global equal latitude/longitude grid/
>
> /projection:/
>
> /init: EPSG:4326/
>
> /shape:/
>
> /width: 21600/
>
> /height: 10800/
>
> /area_extent:/
>
> /lower_left_xy: [-180.0, -90.0]/
>
> /upper_right_xy: [180.0, 90.0]/
> 2.The enhancements are configured exactly in the way you proposed.
>
> No, the images are not was I wanted. I need to get a “mix” of the two
> images like those I sent here (possibly for a European domain – I can
> change that with no problem) that look like the one I sent in my first
> e-mail and reattach now. As you can see, my scripts work with no problem
> when I use the two composites ('clouds_with_background_day' and
> 'night_ir_with_background_hires') separately.
>
> Up to now the related lines of my script are shown below. If anyone
> could help I would be grateful. I think we are very close to succeed!
>
> */Parts of script/*
>
> /composite_1 = 'clouds_with_background_day'/
>
> /composite_2 = 'night_ir_with_background_hires'/
>
> /area = 'Europe_merc'/
>
> /area_def = get_area_def(area)/
>
> /global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')/
>
> /global_scene.load([composite_1,composite_2])/
>
> /#resample_method = 'nearest'/
>
> /#new_scene = global_scene.resample(area, resampler = resample_method,
> cache_dir='/home/thankar/Pytroll/cache_dir_2')/
>
> /resample_method = 'bilinear'/
>
> /new_scene = global_scene.resample(area, radius_of_influence=10000,
> resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir_2')/
>
> /from satpy.composites import DayNightCompositor/
>
> /compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)/
>
> /composite = compositor([new_scene['clouds_with_background_day'],
> new_scene['night_ir_with_background_hires']])/
>
> /new_scene['dnc'] = composite/
>
> /new_scene.save_dataset('dnc', 'MSG4.png')/
>
> *when I use the “nearest” resampler the script gives the following
> output and produces the incorrect day-night image I sent earlier. *
>
> //usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning:
> calling yaml.load() without Loader=... is deprecated, as the default
> Loader is unsafe. Please read https://msg.pyyaml.org/load for full details./
>
> /data = yaml.load(f.read()) or {}/
>
> /The following datasets were not created and may require resampling to
> be generated: DatasetID(name='clouds_with_background_day',
> wavelength=None, resolution=None, polarization=None, calibration=None,
> level=None, modifiers=None),
> DatasetID(name='night_ir_with_background_hires', wavelength=None,
> resolution=None, polarization=None, calibration=None, level=None,
> modifiers=None)/
>
> /Fill value incompatible with integer data using 255 instead./
>
> /Fill value incompatible with integer data using 255 instead./
>
> *When I use the “bilinear” resampler the script crashes with the
> following error.*
>
> //usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning:
> calling yaml.load() without Loader=... is deprecated, as the default
> Loader is unsafe. Please read https://msg.pyyaml.org/load for full details./
>
> /data = yaml.load(f.read()) or {}/
>
> /The following datasets were not created and may require resampling to
> be generated: DatasetID(name='clouds_with_background_day',
> wavelength=None, resolution=None, polarization=None, calibration=None,
> level=None, modifiers=None),
> DatasetID(name='night_ir_with_background_hires', wavelength=None,
> resolution=None, polarization=None, calibration=None, level=None,
> modifiers=None)/
>
> /Traceback (most recent call last):/
>
> /File "msg4_Europe_merc_daynight.py", line 197, in <module>/
>
> /new_scene = global_scene.resample(area, radius_of_influence=10000,
> resampler = resample_method,
> fill_value=0,cache_dir='/home/thankar/Pytroll/cache_dir_2')/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 1105, in resample/
>
> /reduce_data=reduce_data, **resample_kwargs)/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line
> 1060, in _resampled_scene/
>
> /res = resample_dataset(dataset, destination_area, **kwargs)/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 1305, in resample_dataset/
>
> /new_data = resample(source_area, dataset, destination_area,
> fill_value=fill_value, **kwargs)/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 1268, in resample/
>
> /res = resampler_instance.resample(data, **kwargs)/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 418, in resample/
>
> /return self.compute(data, cache_id=cache_id, **kwargs)/
>
> /File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line
> 895, in compute/
>
> /output_shape=target_shape)/
>
> /File
> "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py",
> line 195, in get_sample_from_bil_info/
>
> /res = DataArray(res, dims=data.dims, coords=self.out_coords)/
>
> /File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line
> 229, in __init__/
>
> /coords, dims = _infer_coords_and_dims(data.shape, coords, dims)/
>
> /File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line
> 84, in _infer_coords_and_dims/
>
> /'coordinate %r' % (d, sizes[d], s, k))/
>
> /ValueError: conflicting sizes for dimension 'bands': length 4 on the
> data but length 2 on coordinate 'bands'/
>
>
> Sakis
>
> //
> >      >     <mailto:pytroll%25252Bun...@googlegroups.com
> <mailto:pytroll%2525252Bu...@googlegroups.com>
> >     <mailto:pytroll%2525252Bu...@googlegroups.com
> <mailto:pytroll%252525252B...@googlegroups.com>>>>>.
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com
> >      >      >
> >      >      >      >
> >      >      >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z4Qtp%3Dm%3D4FAZYfPBu0pZbhoHs%3DPsGypUiqvZhQHeh6xg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >      >      >
> >      >      >     --
> >      >      >     You received this message because you are
> subscribed
> >     to the
> >      >     Google
> >      >      >     Groups "pytroll" group.
> >      >      >     To unsubscribe from this group and stop receiving
> >     emails from it,
> >      >      >     send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>
> >      >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>
> >      >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>
> >     <mailto:pytroll%25252Bun...@googlegroups.com
> <mailto:pytroll%2525252Bu...@googlegroups.com>>>>.
> >      >      >     To view this discussion on the web, visit
> >      >      >
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.
> >      >      >
> >      >      > --
> >      >      > You received this message because you are subscribed to
> >     the Google
> >      >      > Groups "pytroll" group.
> >      >      > To unsubscribe from this group and stop receiving
> emails
> >     from it,
> >      >     send
> >      >      > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>
> >      >      > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>>.
> >      >      > To view this discussion on the web, visit
> >      >      >
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com
> >      >
> >      >      >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7yGvHc%2BzejNxF2doeF9CApaBRyYiaeTAR-eQr3JtwWLWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >      >
> >      >     --
> >      >     You received this message because you are subscribed
> to the
> >     Google
> >      >     Groups "pytroll" group.
> >      >     To unsubscribe from this group and stop receiving
> emails from it,
> >      >     send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>
> >     <mailto:pytroll%252Buns...@googlegroups.com
> <mailto:pytroll%25252Bun...@googlegroups.com>>>.
> >      >     To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/b096eea7-c5bd-fbd3-85df-ee1ed4b276d9%40gmail.com.
> >      >
> >      > --
> >      > You received this message because you are subscribed to
> the Google
> >      > Groups "pytroll" group.
> >      > To unsubscribe from this group and stop receiving emails
> from it,
> >     send
> >      > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>
> >      > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>>.
> >      > To view this discussion on the web, visit
> >      >
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com
> >
> >      >
> >
>  <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7xfzYHESdNTPfoOCbg_u2Jt_EaHxseSNbLtBSCsaGa7cQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >
> >     --
> >     You received this message because you are subscribed to the
> Google
> >     Groups "pytroll" group.
> >     To unsubscribe from this group and stop receiving emails from it,
> >     send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> >     <mailto:pytroll%2Bunsu...@googlegroups.com
> <mailto:pytroll%252Buns...@googlegroups.com>>.
> >     To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/f1e63fe6-818b-d120-5ef8-a1363a9e1e16%40gmail.com.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pytroll" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>
> > <mailto:pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z7BCWsoYkJnotFS8Sk1dxP0nJRcJ%3DZEUO4pqqhym-NCA%40mail.gmail.com
>
> >
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7z7BCWsoYkJnotFS8Sk1dxP0nJRcJ%3DZEUO4pqqhym-NCA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pytroll+u...@googlegroups.com
> <mailto:pytroll%2Bunsu...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/ea50203f-76d8-7f2d-9ac2-d785550ffe7d%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pytroll" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/CAJ5NJ7waWmpFRD1vP3hZ4jPUDw9EKtaJS7dCNKQ5gU2Y6e67Xw%40mail.gmail.com
> <https://groups.google.com/d/msgid/pytroll/CAJ5NJ7waWmpFRD1vP3hZ4jPUDw9EKtaJS7dCNKQ5gU2Y6e67Xw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thanasis Karagiannidis

unread,
Apr 24, 2020, 10:31:26 AM4/24/20
to pyt...@googlegroups.com
Dave, to clarify:

1. I did change static_night to static_day. As you expected, nothing changed.
2. I run the script twice. Once with nearest resampler (I turn the bilinear resampling lines to comments) and got the warning about Fill value and once with the bilinear resampler (I turn the nearest resampler lines to comments) and got the error that crashed the script.

Thank you for asking Panu to take a look. I really hope that the problem is in the interpolations. I will wait. Not a problem at all.

Sakis



To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/dcab520a-2dc7-89fb-ec3a-9cd9f8a65744%40gmail.com.

Yufei

unread,
Apr 24, 2020, 2:03:59 PM4/24/20
to pytroll
Hi Sakis,

I don't know why your script did now work, I will leave that to Dave and Panu. :)

However, if you original goal is to generate a ir clouds image over a day/nigh background, I have had another approach, which is working for me. I use the ir_cloud_day as the forground composite and use the day_night composite along with NASA blue marble and black marble static images generating a day/night background on the fly.

I defined my 'clouds_with_backgroud' and the 'day_night_background' composites as below:
 


 clouds_with_background:
    compositor: !!python/name:satpy.composites.BackgroundCompositor
    standard_name: clouds_with_background
    prerequisites:
      - ir_cloud_day
      - day_night_background

day_night_background:
    compositor: !!python/name:satpy.composites.DayNightCompositor
    standard_name: day_night_background
    prerequisites:
      - compositor: !!python/name:satpy.composites.StaticImageCompositor
        standard_name: day_background
        filename: world.topo.200404.3x3600x1800.png
        area: EPSG_4326_3600x1800
      - compositor: !!python/name:satpy.composites.StaticImageCompositor
        standard_name: night_background
        filename: BlackMarble_2016_01deg.jpg
        area: EPSG_4326_3600x1800


I then load the 'clouds_with_background' composite in my script and do the resample and save image, you can find one example from GOES-16 FullDisk scan in the attachement.

Moreover, as you can see I used the low-res marble images, you can define your own with the hi-res  images as you like.


Hope this helps.

Yufei

EPSG_4326_3600x1800_clouds_with_background.tif

Yufei

unread,
Apr 24, 2020, 2:20:39 PM4/24/20
to pytroll

I am sorry that it seems the last attachment is corrupted, here is another try.

7C59867E-C158-4DDB-A8F9-F071A84C12FE_1_105_c.jpeg

Thanasis Karagiannidis

unread,
Apr 24, 2020, 2:55:13 PM4/24/20
to pyt...@googlegroups.com
Dear Yufei

Thank you very much! It worked perfectly!!!

Best regards

Sakis

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.

Yufei

unread,
Apr 24, 2020, 2:57:15 PM4/24/20
to pytroll
Glad I can help.

Yufei
To unsubscribe from this group and stop receiving emails from it, send an email to pyt...@googlegroups.com.

Christian Peters

unread,
Apr 24, 2020, 3:03:03 PM4/24/20
to pytroll
Hi Sakis,

Yufei just post his way to do.

nearest is working here,

new_scene = global_scene.resample(area, radius_of_influence = 20000)

and bilinear needs fill value indeed:

new_scene = global_scene.resample(area, resampler='bilinear', cache_dir='/tmp', fill_value=0)

Bilinear allocates to much memory here to get it working with this high_res background, so I used KDTree.
You don't do it with composites only in your scripts, maybe that makes some trouble?

My composite:

 
 clouds_with_background:
    compositor
: !!python/name:satpy.composites.BackgroundCompositor
    standard_name
: clouds_with_background_night
    prerequisites
:
     
- ir_cloud_day
     
- compositor: !!python/name:satpy.composites.DayNightCompositor
        prerequisites
:
         
- static_day
         
- static_night
 which works here, see pics.

Regards,

Christian
MSG-3-euro-raw-2.jpg
MSG-3-euro-raw-1.jpg

Thanasis Karagiannidis

unread,
Apr 24, 2020, 3:07:55 PM4/24/20
to pyt...@googlegroups.com
I will test this one too.

Thank you very much Christian!

Sakis

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/746883eb-91a5-4cb4-9d76-bef364eef042%40googlegroups.com.

Christian Peters

unread,
Apr 24, 2020, 3:08:29 PM4/24/20
to pytroll
Hi Sakis, hi Yufei,

ok...I'm to slow....already solved....! :-D

That's fine!

Regards,

Christian


Reply all
Reply to author
Forward
0 new messages