Dear all
I wish to make an image like the following (but maybe for a smaller domain) using msg hrit data and 2 static backgrounds (one for day and one for night). I tried the DayNightCompositor but it doesn’t seem to work.
I think I do not have the correct compositors etc
Can anyone direct me on this?
Sakis
--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/9fa7329d-c2ba-bad0-d7ed-4560e0cca842%40fmi.fi.
Dave I tried to load and resample the 2 global scenes. The fist one is resampled just fine but the second one crashes. Should I somehow “delete” the first global scene before moving to the next one? I include my part of the code and the outputs
Related part of script
composite_1 = 'clouds_with_background_day'
composite_2 = 'night_ir_with_background_hires'
area = 'Europe_merc'
files = glob(tmpdir + "/" + "H-000-MSG*" + Dat + "*")
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
# Loading and resampling comp 1
global_scene.load([composite_1])
resample_method = 'bilinear'
new_scene_1 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir')
print("ok scene 1")
# Loading and resampling comp 2
global_scene.load([composite_2])
resample_method = 'bilinear'
new_scene_2 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir')
print("ok scene 2")
from satpy.composites import DayNightCompositor
compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)
composite = compositor([new_scene_1['clouds_with_background_day'], new_scene_2['night_ir_with_background_hires']])
Output of run
/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f.read()) or {}
The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
ok scene 1
The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Traceback (most recent call last):
File "msg4_Europe_merc_daynight.py", line 196, in <module>
new_scene_2 = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir')
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample
reduce_data=reduce_data, **resample_kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene
res = resample_dataset(dataset, destination_area, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset
new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample
res = resampler_instance.resample(data, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample
return self.compute(data, cache_id=cache_id, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute
output_shape=target_shape)
File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info
res = DataArray(res, dims=data.dims, coords=self.out_coords)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__
coords, dims = _infer_coords_and_dims(data.shape, coords, dims)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims
'coordinate %r' % (d, sizes[d], s, k))
ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'
Sakis
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/d93e7b62-3f3d-5752-08f1-418d58a964db%40gmail.com.
I tried
global_scene.load([composite_1,composite_2])
but the error continued
/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f.read()) or {}
The following datasets were not created and may require resampling to be generated: DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Traceback (most recent call last):
File "msg4_Europe_merc_daynight.py", line 191, in <module>
new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir')
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample
reduce_data=reduce_data, **resample_kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene
res = resample_dataset(dataset, destination_area, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset
new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample
res = resampler_instance.resample(data, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample
return self.compute(data, cache_id=cache_id, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute
output_shape=target_shape)
File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info
res = DataArray(res, dims=data.dims, coords=self.out_coords)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__
coords, dims = _infer_coords_and_dims(data.shape, coords, dims)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims
'coordinate %r' % (d, sizes[d], s, k))
ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'
BUT, when I changed my resampling method to nearest
resample_method = 'nearest'
new_scene = global_scene.resample(area, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir')
the script continued but crashed later, when I tried to save the dataset
Traceback (most recent call last):
File "msg4_Europe_merc_daynight.py", line 275, in <module>
new_scene.save_dataset(composite,'./MSG4.png')
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1295, in save_dataset
return writer.save_dataset(self[dataset_id],
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 681, in __getitem__
return self.datasets[key]
File "/usr/local/lib/python3.7/dist-packages/satpy/readers/__init__.py", line 307, in __getitem__
return super(DatasetDict, self).__getitem__(item)
TypeError: unhashable type: 'DataArray'
So, firstly for some reason the resampling using bilinear fails although it has no problem when used in the exact same way in separate scripts. Could it be something related to the projections or resolutions of the specific composites? To support that, when I replace these composites with “day_microphysics” and “night_microphysics” no resampling is required before I use the DayNightCompositor (I tried that). But besides that, why does the new_scene.save_dataset(composite,'./MSG4.png') fails?
Sakis
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/b096eea7-c5bd-fbd3-85df-ee1ed4b276d9%40gmail.com.
When I added
new_scene['dnc'] = composite
new_scene.save_dataset('dnc', 'MSG4.png')
the image was saved. But something is still wrong. With fill_value=0 in resampling (nearest) I got the image FV0.jpg while without fill_value=0 sapty set 255 for fill values (“Fill value incompatible with integer data using 255 instead.”) and I got image FV255.jpg. As for the clearing of the cache directory I tried it but it did not helped.
Sakis
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/f1e63fe6-818b-d120-5ef8-a1363a9e1e16%40gmail.com.
Ok, a lot of issues! To summarize:
1. The “clouds_with_background_day” was created following the example of Christian Peters (mail entitled “colorize image”) with the following
seviri.yaml
clouds_with_background_day:
compositor: !!python/name:satpy.composites.BackgroundCompositor
standard_name: clouds_with_background_day
prerequisites:
- ir_cloud_day
- static_day
static_day:
compositor: !!python/name:satpy.composites.StaticImageCompositor
standard_name: static_night
filename: /xyz/satpy/backgrounds/world.200407.3x21600x10800.png
area: EPSG_4326
generic.yaml
static_day:
standard_name: static_day
operations:
- name: stretch
method: *stretchfun
kwargs:
stretch: crude
min_stretch: [0, 0, 0]
max_stretch: [255, 255, 255]
areas.yaml
EPSG_4326:
description: Global equal latitude/longitude grid
projection:
init: EPSG:4326
shape:
width: 21600
height: 10800
area_extent:
lower_left_xy: [-180.0, -90.0]
upper_right_xy: [180.0, 90.0]
while I downloaded and put in a directory the
https://neo.sci.gsfc.nasa.gov/archive/bluemarble/bmng/world_2km/world.200407.3x21600x10800.png
2. The enhancements are configured exactly in the way you proposed.
No, the images are not was I wanted. I need to get a “mix” of the two images like those I sent here (possibly for a European domain – I can change that with no problem) that look like the one I sent in my first e-mail and reattach now. As you can see, my scripts work with no problem when I use the two composites ('clouds_with_background_day' and 'night_ir_with_background_hires') separately.
Up to now the related lines of my script are shown below. If anyone could help I would be grateful. I think we are very close to succeed!
Parts of script
composite_1 = 'clouds_with_background_day'
composite_2 = 'night_ir_with_background_hires'
area = 'Europe_merc'
area_def = get_area_def(area)
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite_1,composite_2])
#resample_method = 'nearest'
#new_scene = global_scene.resample(area, resampler = resample_method, cache_dir='/home/thankar/Pytroll/cache_dir_2')
resample_method = 'bilinear'
new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir_2')
from satpy.composites import DayNightCompositor
compositor = DayNightCompositor("dnc", lim_low=85., lim_high=88.)
composite = compositor([new_scene['clouds_with_background_day'], new_scene['night_ir_with_background_hires']])
new_scene['dnc'] = composite
new_scene.save_dataset('dnc', 'MSG4.png')
when I use the “nearest” resampler the script gives the following output and produces the incorrect day-night image I sent earlier.
/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f.read()) or {}
The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Fill value incompatible with integer data using 255 instead.
Fill value incompatible with integer data using 255 instead.
When I use the “bilinear” resampler the script crashes with the following error.
/usr/lib/python3/dist-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f.read()) or {}
The following datasets were not created and may require resampling to be generated: DatasetID(name='clouds_with_background_day', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='night_ir_with_background_hires', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Traceback (most recent call last):
File "msg4_Europe_merc_daynight.py", line 197, in <module>
new_scene = global_scene.resample(area, radius_of_influence=10000, resampler = resample_method, fill_value=0, cache_dir='/home/thankar/Pytroll/cache_dir_2')
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1105, in resample
reduce_data=reduce_data, **resample_kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/scene.py", line 1060, in _resampled_scene
res = resample_dataset(dataset, destination_area, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1305, in resample_dataset
new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 1268, in resample
res = resampler_instance.resample(data, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 418, in resample
return self.compute(data, cache_id=cache_id, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/satpy/resample.py", line 895, in compute
output_shape=target_shape)
File "/usr/local/lib/python3.7/dist-packages/pyresample/bilinear/xarr.py", line 195, in get_sample_from_bil_info
res = DataArray(res, dims=data.dims, coords=self.out_coords)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 229, in __init__
coords, dims = _infer_coords_and_dims(data.shape, coords, dims)
File "/usr/lib/python3/dist-packages/xarray/core/dataarray.py", line 84, in _infer_coords_and_dims
'coordinate %r' % (d, sizes[d], s, k))
ValueError: conflicting sizes for dimension 'bands': length 4 on the data but length 2 on coordinate 'bands'
Sakis
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/ea50203f-76d8-7f2d-9ac2-d785550ffe7d%40gmail.com.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/dcab520a-2dc7-89fb-ec3a-9cd9f8a65744%40gmail.com.
I am sorry that it seems the last attachment is corrupted, here is another try.

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/0931022e-abb7-424b-9606-0fb04310c3ec%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to pyt...@googlegroups.com.
new_scene = global_scene.resample(area, radius_of_influence = 20000)new_scene = global_scene.resample(area, resampler='bilinear', cache_dir='/tmp', fill_value=0)
clouds_with_background:
compositor: !!python/name:satpy.composites.BackgroundCompositor
standard_name: clouds_with_background_night
prerequisites:
- ir_cloud_day
- compositor: !!python/name:satpy.composites.DayNightCompositor
prerequisites:
- static_day
- static_night--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/746883eb-91a5-4cb4-9d76-bef364eef042%40googlegroups.com.