Known circular artifacts wih TrollFlow2 (composites wih HRV)

[José Daniel Gómez de Segura]

Hi,

We have started processing downloaded MSG data from Eumetsat's Data Store and understanding trollflow2 to obtain images.

Unfortunately, our final products have to be reprojected to a reference system that can be understood by Geoserver. Therefore, we wrote a new area definition in areas.yaml:

EuropeCanaryHRV:
  description: trial area
  projection:
    proj: lonlat
    ellps: WGS84
  shape:
    height: 4800
    width: 9000
  area_extent:
    lower_left_xy: [-44.0, 26.0]
    upper_right_xy: [35.0, 65.0]
    units: deg

This area is used for composites with the HRV channel, like ir_sandwich. The outputs have the known circular artifacts, like the attached image.

We have already tried with radius_of_influence up to 80000 (defined in trollflow2.yaml), and the problem isn't there when repeating the procedure from a standalone python script, like this one:

global_scene = Scene(reader="seviri_l1b_native", filenames=[filenames[20]])

composite='ir_sandwich'

global_scene.load([composite])

lscene = global_scene.resample("test7", radius_of_influence=80000)

lscene.save_dataset(composite, './test/irsand_latlon.tif')

The HRV channel alone is also rendered correctly by trollflow2. I suspect he problem arises when HRV is mixed with the rest of standard resolution channels.

I was wondering if someone has found the same issue before and could share a solution, in which case I'd be really grateful.

Extra question: is there a way to pydecorate or pycoast an image from trollflow2? I don't mind investigating, but I'm at a loss here and don't know really where to start from.

Thanks a lot!

Regards,

José Daniel Gómez de Segura

Meteorology and Climate Area

Tecnalia

[Raspaud Martin]

Dear José,

Thanks a lot for contacting us regarding this issue. I will try to
reproduce the problem shortly and report back here.

Best regards,
Martin

[José Daniel Gómez de Segura]

That's great, Martin.

Thanks!

José

--
You received this message because you are subscribed to a topic in the Google Groups "pytroll" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pytroll/8iHh1pooBSc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/f7c1d97e1c08ea9dbe799b1986c068e354c381c7.camel%40smhi.se.

[Raspaud Martin]

Dear José,

Thanks for the providing the script and area to reproduce the problem, it allowed me to test quickly with the data I have locally.

Unfortunatelly I don't have seviri data in native format at hand, so I tried with a sample in hrit format instead, and it seems to be working fine, seen attached (reduced) image.

I will try to get my hands on a native file now, but could you tell us what versions of satpy and pyresample you are using in the meanwhile?

Best regards,

Martin

[José Daniel Gómez de Segura]

Dear Martin,

Sure! I forgot to add that information. Here are some of the packages we have

dask                      2021.12.0          
dask-core                 2021.12.0        
decorator                 5.1.0            
distributed               2021.12.0       
posttroll                 1.6.1                  
proj                      8.2.0                
pycoast                   1.5.0          
pyresample                1.22.3         
pyspectral                0.10.5             
python                    3.9.7          
pytroll-collectors        0.11.1                   
pytroll-schedule          0.5.2                
satpy                     0.31.0             
trollflow2                0.11.1                   
trollimage                1.17.0             
trollmoves                0.9.0                    
trollsift                 0.3.5              
xarray                    0.20.2             

Regards,

José

You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/8139f71507462309f0d275e16f08d1f3dadd3164.camel%40smhi.se.

[Raspaud Martin]

Dear José,

Thanks for the version. From what I see what you get comes from the
radius_of_influence not being applied correctly in trollflow2. Would
you mind sharing the product list you are using? The yaml syntax is
sensitive to indentation, so make sure you double check that the
radius_of_influence parameter is at the first indentation level after
"product_list". As a reference, here is the beginning of ours (although
we do not produce that particular product):

```
product_list:
output_dir: /san1/geo_out/rss/imagery
use_extern_calib: false
fname_pattern:
"{platform_name}_{start_time:%Y%m%d_%H%M}_{area}_{product}.{format}
"
publish_topic: /raster/2A/seviri/rss
reader: seviri_l1b_hrit
min_coverage: 25
use_tmp_file: true
delay_composites: true
mask_area: false
radius_of_influence: 20000
reduce_data: false
cache_dir: "/var/tmp/"
```

Looking further into this now.

Best regards,
Martin

[Raspaud Martin]

Dear José,

In the trollflow2 log, you should have a line like this:
[DEBUG: 2021-12-17 16:11:14 : trollflow2.plugins] area: eurol,
area_conf: {'radius_of_influence': 80000, 'resampler': 'nearest',
'reduce_data': False, 'cache_dir': '/var/tmp/', 'mask_area': False,
'epsilon': 0.0}

There you can see if trollflow2 takes into account you settings.

I have tried reproducing the problem here, but it seems that the radius
of influence is correctly use, so I get a correct image with 80000.

Best regards,
Martin

[José Daniel Gómez de Segura]

Dear Martin,

Thanks for your suggestions. I suspect you nailed it, something in the yaml file, because we use two areas, one is using radius_of_influence and the other is not (None in the logs).

[DEBUG: 2021-12-18 09:45:48 : trollflow2.plugins] area: EuropeCanaryHRV, area_conf: {'radius_of_influence': None, 'resampler': 'nearest', 'reduce_data': True, 'cache_dir': None, 'mask_area': False, 'epsilon': 0.0}

[DEBUG: 2021-12-20 07:44:06 : trollflow2.plugins] area: EuropeCanary, area_conf: {'radius_of_influence': 80000, 'resampler': 'nearest', 'reduce_data': True, 'cache_dir': None, 'mask_area': False, 'epsilon': 0.0}

After a careful inspection of the file, it turns out that there was a typo in radius_of_influence for the HRV area. It's been corrected and all is working, even with a more sensible radius of 20000.

My sincere apologies for my mistake and deep appreciation for your support.

Best regards,

José

To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/b0b0e33dfb33c4093678037c9a2b2bcfa6b50d22.camel%40smhi.se.

[Raspaud Martin]

Dear José,

Great that you found the problem!

No need to apologise, it's always good with feedback to make sure
things are working as expected :)

Best regards,
Martin