reading MTAST HRIT data

490 views
Skip to first unread message

ioan.f...@geomodel.eu

unread,
Mar 27, 2015, 1:03:19 PM3/27/15
to pyt...@googlegroups.com
Hello folks
I would to know it pytrol can read MTSAT data in HRIT format (broadcasted) not fomr eumetsat
The file name has the following name
 IMG_DK01IR1_201310101032_001

where DK01 = full disk sscan type
IR1 = band
201310101032 = datem time
001 -spatial segment of scan (1-10)

and if so how would the config look like

would something like this suffice?

[satellite]
satname = 'mtsat'
number = '2'
instruments = ('mviri',)
projection = 'geos(145.0)'

[mviri-level2]
format = 'mipp_xrit'

[mviri-level1]
format = 'mipp/xrit/SGS'
dir = '/home/jano/Desktop/mtsat/DK01_201310101030'
#filename = 'L-000-MSG?__-MTSAT2______-%(channel)s_145E-%(segment)s-%Y%m%d%H%M-__'
filename = 'IMG_DK01%(channel)_%Y%m%d%H%M_%(segment)'

[mviri-1]
name = '00_7'
frequency = (0.5, 0.7, 0.9)
resolution = 4000.0
size = (2752, 2752)

[mviri-2]
name = '03_8'
frequency = (2.8, 3.8, 4.8)
resolution = 4000.0
size = (2752, 2752)

[mviri-3]
name = '06_8'
frequency = (6.1, 6.8, 7.5)
resolution = 4000.0
size = (2752, 2752)

[mviri-4]
name = '10_8'
frequency = (9.8, 10.8, 11.8)
resolution = 4000.0
size = (2752, 2752)

thanks in advance

Raspaud Martin

unread,
Mar 28, 2015, 2:56:06 AM3/28/15
to pyt...@googlegroups.com
Hello Ioan,

I don't think we have tested mipp on this kind of data, so you're very welcome to try it out and give us some feedback. Regarding the config file, it looks good as far as I can see.

Best regards,
Martin

Skickat från min HTC
--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Lars Orum Rasmussen

unread,
Mar 29, 2015, 5:30:47 AM3/29/15
to pyt...@googlegroups.com
Hi Ioan,

Yes, mipp can read broadcasted MTSAT data. It's a very old interface, and, I believe, was never integrated into mpop. The same interface is used to read MET7 and GOES. Here at DMI, we are using the following control file:

#
# Level 1.5 configuration file for MTSAT2
#
# An item like:
#   name = value
# is read in python like:
#   try:
#       name = eval(value)
#   except:
#       name = str(value)
#

[satellite]
satname = 'mtsat'
number = '2'
instruments = ('mviri',)
projection = 'geos(145.0)'

[mviri-level2]
format = 'mipp_xrit'

[mviri-level1]
format = 'mipp/xrit/SGS'
dir = '/data/xrit/out'
filename = 'L-000-MSG?__-MTSAT2______-%(channel)s_145E-%(segment)s-%Y%m%d%H%M-__'

[mviri-1]
name = '00_7'
frequency = (0.5, 0.7, 0.9)
resolution = 4000.0
size = (2752, 2752)

[mviri-2]
name = '03_8'
frequency = (2.8, 3.8, 4.8)
resolution = 4000.0
size = (2752, 2752)

[mviri-3]
name = '06_8'
frequency = (6.1, 6.8, 7.5)
resolution = 4000.0
size = (2752, 2752)

[mviri-4]
name = '10_8'
frequency = (9.8, 10.8, 11.8)
resolution = 4000.0
size = (2752, 2752)

I have attached the code we are using, to read these data files.

Regards, Lars



--
process_fsd

Lars Orum Rasmussen

unread,
Mar 29, 2015, 5:43:33 AM3/29/15
to pyt...@googlegroups.com
By the way, it should be "easy" to read it through MPOP, with the format = 'mipp/xrit/SGS' ... but I'm not sure if it has been tested.

Regards, Lars

2015-03-27 18:03 GMT+01:00 <ioan.f...@geomodel.eu>:

--

ioan.f...@geomodel.eu

unread,
Mar 30, 2015, 4:39:34 AM3/30/15
to pyt...@googlegroups.com
Thanks Lars, Martin,

I have checked  the JMA HRIT specs and I believe it should be possible to read the HRIT with MIPP package.(_xrit.py). Not sure tough is the _xrit reader can handle compressed data because JAM hrits can come compressed. I am still researching these things
Nevertheless, JMA appends its own Header (type 130) where image compensation information is stored and I am after this one
as I need to use those coefficients to correctly register the image. But i first need to do some checking, to see exactly what is the effect of these coefficients on the image registration accuracy. If this will solve my problem
the the way forward could be to create a class in xrit module which would map to JMA HRIT.

Thanks  a lot
Ioan

Martin Raspaud

unread,
Mar 30, 2015, 2:28:06 PM3/30/15
to pyt...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ioan,

Have a look at the feature-noaa-lrit branch on github, there we have
implemented a header 130. Don't know if it's the same as for JMA though.

Best regards,
Martin
> <https://groups.google.com/d/optout>.
>
>
> -- You received this message because you are subscribed to the
> Google Groups "pytroll" group. To unsubscribe from this group and
> stop receiving emails from it, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>. For more options,
> visit https://groups.google.com/d/optout.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVGZW1AAoJEBdvyODiyJI4ax8IAOVCcHfQPm4s5RIFQHUIc2DK
sAvIYWbgRBFeJbe2FVAs87jIGQciWciNIJFSlVunw6cyiY0zhaUWJfAPegqXd5Ro
W1alcNU1vEqWUNA/a62We/lYHaSuNuY8GKgSc3/z22jEG4xo2STRUmJZFrP8uEFv
2aa/ynixRuDZUqmM4CxJU1yfsZiw4AAg4DGMZXgWGFrIidh59zTM3jgbcf/XFpLn
uT8lUsvjjvB7jlNiski12OJtcLalZ6Ds7meZo08w5ocrSOut3K3wSVhxlDbr91xA
gtNTyyTLi+G2J24s9vihg9yiJXFeJqB1PdonjkWyz0XHRkNYJPJPdEvoV/w9aw8=
=Pm2Z
-----END PGP SIGNATURE-----
martin_raspaud.vcf

bill....@gmail.com

unread,
Mar 30, 2015, 8:53:19 PM3/30/15
to pyt...@googlegroups.com
Hi Ioan

We've also been looking at reading the JMA supplied mtsat2 data as part of our work to migrate to Himawari8/9. It's very much a work in progress but I've attached what we've done so far in the tar. (Do note, we've not yet looked at the values in header 130). In the config file you'll need to point to your data dir and ensure the env. var PPP_CONFIG_DIR is set to your cfg directory.

As Martin has already suggested, we have taken some guidance from the feature-noaa-lrit branch and associated thread and also taken some hints from this thread https://groups.google.com/forum/#!topic/pytroll/Rvt4MJhteLc .

We are still reviewing pytroll, so if you use the attached modifications, please treat the attached code with much suspicion and check you are getting good values (for instance we have performed a bytseswap when loading data [mipp/xrit/loader.py]; and there is no prologue file from JMA so we currently use the header information from the first segment file as the prologue, this may impact your use of header 130 if it is different across segments.). 

The changes here assume the northern and southern hemisphere tar.gz files from JMA have already been uncompressed in to the source directory (it might be possible to get mipp to do that but we'll probably script that untar'ing task prior to invoking pytroll, if we go down the road of using pytroll of course).

The changes we have done will also have probably broken eumetsat sourced mtsat2 data in pytroll. We don't have access to that feed, so it probably doesn't matter for us. We also have the issue of ongoing maintenance, e.g. how would we deploy pytroll updates in the future without disrupting our own changes? 

If Martin (or other Pytroll developers) have time to critique what we have done that would be useful.

So we still have a lot to work through but I hope what is here helps you.


For testing pytroll I've been using...

from PIL import ImageOps
import Image # as PIL_Image
from datetime import datetime
from mpop.satellites import GeostationaryFactory
import numpy as np
from pyresample.geometry import AreaDefinition
t = datetime(2015, 03, 25, 01, 32)
mtsat = GeostationaryFactory.create_scene("mtsat", "2", "mviri", time_slot=t)
mtsat.load([0.7, 3.8, 6.8, 10.8, 12.00]) #, 3.8, 6.8, 10.8])
print mtsat

globe = mtsat.image.channel_image(0.7)
img = globe.pil_image()
from pycoast import ContourWriter
cw = ContourWriter('/opt/spare/gshhg')
proj4_string = '+proj=geos +lon_0=145 +a=6378169.00 +b=6356583.80 +h=35785831.0'
area_extent =  (-5499500.0, -5500500.0, 5500500.0, 5499500.0)
area = (proj4_string, area_extent)
cw.add_coastlines(img, area, resolution='h', level=5, outline='white')
cw.add_grid(img, area, (10.0,10.0),(2.0,2.0), fill='white',
             outline='white', minor_outline='white')
img.save("./vis_globe_coast.png")

In case you're looking at resampling/projecting here are a couple of area.def specifications

REGION: TROPICS {
         NAME:          TROPICS
         PCS_ID:        tropics
         PCS_DEF:       proj=merc,lat_ts=0,a=6370997.0,lon_0=0,lat_0=0
         XSIZE:         1344
         YSIZE:         848
        AREA_EXTENT:   (-555555.4924,-4156086.2753,7779110.2268,1102690.9047)
};

REGION: NZ_LL {
         NAME:          NZ_LL
         PCS_ID:        nz_ll
         PCS_DEF:       proj=eqc,lat_ts=0,a=6370997.0,lon_0=0,lat_0=0
         XSIZE:         840
         YSIZE:         540
         AREA_EXTENT:   (-1111110.9848,-6666665.9088,6666665.9088,-1666666.4772)
}

We've found the re-projecting quite challenging.

Best regards
Bill Self
MetService NZ.
pytroll_bespoke.tar

ioan.f...@geomodel.eu

unread,
Mar 31, 2015, 3:49:48 AM3/31/15
to pyt...@googlegroups.com
Hello Bill,
thanks for your comprehensive  reply. 

I'll try your version and let you know the outcome.
About the 130 Header, theoretically it varies across space/segments so an implementation of "Image Compensation Information Header" (130) header is required.


Ioan

ioan.f...@geomodel.eu

unread,
Mar 31, 2015, 3:49:51 AM3/31/15
to pyt...@googlegroups.com, martin....@smhi.se
Hi Martin,
I have checked the header.....i tend to believe it is different.
THe JMA HRIT Header 130  reads like this
Header Type #130 - Image Compensation Information Header
            Header_Record_Length : 2019
  Image_Compensation_Information : 
LINE:=1
COFF:=5500.1
LOFF:=5500.0
LINE:=21
COFF:=5500.1
LOFF:=5500.0
LINE:=41
COFF:=5500.1
LOFF:=5500.0
LINE:=61
COFF:=5500.1
LOFF:=5500.0
LINE:=81
COFF:=5500.1
LOFF:=5500.0
LINE:=101
COFF:=5500.1
LOFF:=5500.0
LINE:=121
COFF:=5500.1
LOFF:=5500.0
LINE:=141
COFF:=5500.1
LOFF:=5500.0
LINE:=161
COFF:=5500.1
LOFF:=5500.0
LINE:=181
COFF:=5500.1
LOFF:=5500.0
LINE:=201
COFF:=5500.1
LOFF:=5500.0
LINE:=221
COFF:=5500.1
LOFF:=5500.0
LINE:=241
COFF:=5500.1
LOFF:=5500.0
LINE:=261
COFF:=5500.1
LOFF:=5500.0
LINE:=281
COFF:=5500.1
LOFF:=5500.0
LINE:=301
COFF:=5500.1
LOFF:=5500.0

Anyhow, I will 

ioan.f...@geomodel.eu

unread,
Apr 1, 2015, 6:17:14 AM4/1/15
to pyt...@googlegroups.com
Hello Bill,

i managed to read the broadcasted JMA HRIT files. Without few exceptions things went smoothly.
I changed the computation of subsatellite point in SGS.py.read_metadata function as it was incorrectly computing the ssp and then assigning it to the area_def object of the Image


now some questions for Martin, Lars (maybe too many because of my ignorance):)


The JMA header 130 contains adjusted COFFs and LOFFs for each line/segment file. How could i use these update values when re-projecting the HRIT?..let's say to LATLON  or when displaying the IMAGE using basemap
Intuitively I would resort to some image  geometry class (perhaps pyresample.geomtery.AreaDefinition)...ssubslass it and use those values.

I have limited knowledge of HRIT format. The HRIT docs talk about three type coordinates (LAT/LON, COLUMN/LINE) and X/Y. Are XY the geos(14[05]) projection coordinates?

I suspect packages like pyproj can handle a transformation form geos(140) to LL and viceversa but it is not clear to me how to hook the scaling factors into the transformation in context of pyresample

Thanks again for you help guys.






On Tuesday, March 31, 2015 at 2:53:19 AM UTC+2, bill....@gmail.com wrote:

bill....@gmail.com

unread,
Apr 1, 2015, 5:22:21 PM4/1/15
to pyt...@googlegroups.com
Hi Ioan

D'uh, I see what I did there with ssp. Thanks for the pointer, it was probably feeding in to why we found reprojecting awkward.

Good to hear you're loading data now. If you work out how best to use header 130 we would be interested in seeing how you use that data, if possible.

Best regards
Bill

Martin Raspaud

unread,
Apr 2, 2015, 1:44:07 AM4/2/15
to pyt...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ioan,

the LOFF and COFF parameters are used to skip proj.4 to generate the
lons and lats directly, but as you said proj.4 has the "geos"
projection defined which could be used in this case. And actually if
you use mipp through mpop, the AreaDefinition object is generated on
the fly, so you shouldn't have to do anything.

Best regards,
Martin
> <https://groups.google.com/forum/#%21topic/pytroll/Rvt4MJhteLc> .
> -- You received this message because you are subscribed to the
> Google Groups "pytroll" group. To unsubscribe from this group and
> stop receiving emails from it, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>. For more options,
> visit https://groups.google.com/d/optout.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVHNclAAoJEBdvyODiyJI4tXcIAMzsaPTNkwrN2DPDBvL4FlPp
w+u6TovLWjIUzbGI5GRNqiz7f/4ljJ0WpIX2k7uY2b68o/cu1NUl5TDpwY18GvKz
WvblfqWvPjGJg5jHGAwlrjn2VwHqnXN3WJfhaO7MI4QUzPoAqi6B+TPZyFg1Eztb
7SN6qYgOvWTl/sxqo8LnprYeSJQpTtRwhyXWUI4u7msbfFdoCiPsR6Rnt+MCH/MP
jIjvlUmtYpUZS1sdZKupd/VbDV+pPycld0tQtMj6FfAyIb9qnTgo7p/5WboxHpdd
p2ZOhElttFrQx/mo1l7MaQgm/dB0eNeHlnReXnyJRhBin9vRUkNqmxVH4zLyc1M=
=LMmM
-----END PGP SIGNATURE-----
martin_raspaud.vcf

ioan.f...@geomodel.eu

unread,
Apr 2, 2015, 7:37:35 AM4/2/15
to pyt...@googlegroups.com
Bill,
i'll let you know for sure as soon as things progress

ioan.f...@geomodel.eu

unread,
Apr 2, 2015, 8:16:16 AM4/2/15
to pyt...@googlegroups.com, martin....@smhi.se
Martin,

as far as I am concerned 

according to  CGMS the  reprojection/transformation from JMA HRIT image row/col to lat/lon uses the intermediary coordinates
(x,y). These coordinates are expressed in radians (which makes sense as the satellite uses angles). Further, the CFAC/LFAC and COFF/LOFF
are used to scale(unpack if you wish) the intermediate coordinates and get image/pixel row col.

Although the CFAC/LFAC and COFF/LOFF should be unique for one image/segment, for various reasons(satelite instability I assume) COFF/LOFF vary. This is why JAM has
sampled every 20-th line/column and appended these values in the header 130

So from my point of view the reprojection of JMA HRIT images should use header 130 data. Otherwise the images can shifted in both axes (which is exactly why I a am looking into these things).
When I visualize the HRIT data in basemap i see the islands are misaligned with the coastlines and the misalignment is not systematic. This fact lead me to the conclusion there must be some metadata in the headers that are not used during reprojection.

I myself fail to see the connection between the CFAC/LFAC COFF/LOFF and the geostationary (meters) coordinates. But maybe I am wrong. It could be as well that there is a linear relationship between the geos(140) coords and image coordinates (row/col)

regards
Ioan

Martin Raspaud

unread,
Apr 2, 2015, 10:04:30 AM4/2/15
to ioan.f...@geomodel.eu, pyt...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ioan,

Ok, now I understand. I was supposing the data was geo-rectified as
the one we get from Eumetsat, meaning that L/COFF and L/CFAC do not
vary. Then it's easy to apply the geos projection. Unfortunately, as
far as I know, the geos projection doesn't allow for parameters like
offset and l/cfac. So that means that you have to compute the
lons/lats of each pixel.

Now, the CGMS describes exactly the procedure, and you can probably
use numexpr for extra fast computation (if it's still slow, I can help
you optimize). When you're done, just put the result in a
pyresample.geometry.SwathDefinition object, and you're good to go.

Best regards,
Martin
>> <https://groups.google.com/forum/#%21topic/pytroll/Rvt4MJhteLc
> <https://groups.google.com/d/optout>.
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVHUxtAAoJEBdvyODiyJI4cpwIAN46pGOVttZpdWTOTPVZ7Fjz
VKe7JKHrf+VRotaJjqoF0EeUUvBkkrSfxuhyX2UeFXi2dU8W5MJnJDXsWxP7WMAx
CribGKMsTI2s/3Tsn4y26/9+HMFPN5vzFdSi/BmC1OkSYE/bo2kpIPP7TJ8gaXiN
KsuKGgm4wdcTM8+NibMPPlkZY9A7LXvGaOWJG3pF9IWG7+K5CLU560HKMSk6lgcX
8WPsd4fzE07qV9enr65bHpEErmd0SCzw3PimzuziWV2mD7OgWPEDkarJDVuz1JuU
pHsjBX3jSnMpuM7TsaJWsjLVEtSokwvTwtX7ozI88RPgUa31AU+t8BjJFIjQTfw=
=faz9
-----END PGP SIGNATURE-----
martin_raspaud.vcf

ioan.f...@geomodel.eu

unread,
Apr 14, 2015, 10:05:12 AM4/14/15
to pyt...@googlegroups.com, ioan.f...@geomodel.eu, martin....@smhi.se
Hello Martin, Bill

I have come to  improve my understanding of HRIT/JMA HRIT format.

I have created a reader from scratch. I managed to read all headers and data  (see attached file). 
The data is raw (counts)
I will probably implement a JMA reader in pytroll but first i am getting my hands dirty with reprojection.

I would like to optimize it somehow because it is slow.

In a naive way i am allocating memory for the results in latlon and then computing LINE/COL indices for each pixel in the output and fetching data from the raw file using these indices. I can imagine a more robust way to do this
(NN as pyresample) but i also belive it should be possible to twist the things a bit and do the reprojection for all points at once (i call this matrix mode)


 For this reason I have implemented a couple of functions
to translate the LAT/LON coords into image LINE/COL. In order to do this correctly (each image line with it's own COFF and LOFF) I linearly interpolated COFF and LOFF
for each line in the image.

Bill, in case you wat to give it a shot, reading JMA HRIT files would look like this

#read the data

fp = '/home/jano/Desktop/mtsat/samples/IMG_DK01VIS_200710080130_001'
fpat = fp.replace('001', '*')
hrit_image = JMAHRIT(fpattern=fpat)


#plot

from pylab import plt, show
fig = plt.figure( facecolor='w')
ax = fig.add_subplot(111)
ax.imshow(hrit_image.data, cmap='gray', interpolation='nearest' )
show()




I will get in touch when i sort it out a bit.

Thanks again for your help guys
file.py

ioan.f...@geomodel.eu

unread,
Apr 17, 2015, 7:50:58 AM4/17/15
to pyt...@googlegroups.com
Hi Bill,

i have managed to successfully reproject an JMA HRIT image to LATLON using the COFF/LOFF from the header.
Using tricks like broadcasting reading&reprojecting a full disk VIS image on my machine lasts approx 13 secs.

I can share the code if you are interested.
Ioan

Martin Raspaud

unread,
Apr 17, 2015, 7:57:03 AM4/17/15
to pyt...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Ioan,

I'm interrested in the code for integration to pytroll :)

Best regards,
Martin
> <https://groups.google.com/forum/#%21topic/pytroll/Rvt4MJhteLc> .
> -- You received this message because you are subscribed to the
> Google Groups "pytroll" group. To unsubscribe from this group and
> stop receiving emails from it, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>. For more options,
> visit https://groups.google.com/d/optout.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVMPUNAAoJEBdvyODiyJI4T78IAJrqESrOH8OrPS2htlscDKPw
KsvVkT1nK6nWt/zEhvJNMkvusNYFBvMfo+1Lr4eZVnl9plRbcXbeJKfWFVg8u4Ec
cJf01bLNsG3grE751yRr/rJgjEjgtdp2ad25NsBOH/IHLjEy+oLm6Z5N5qvsd5fB
A3efwdvH5A1Dh43yZONozeUXQAgkWA513WFy1up3z7sikXFi6+4MQeiA+981HgfI
pMSYfGa9nYnvivDDhl+HQKaCHCm/FlOlH6K6pJ/1fZyrTJRXNDzR14A0/8cGimwD
GRPfA3w0JcJE9xSLgX1xZwzRUtaUjVIgpmUmD9fLhoyQqUrAMCTvQsRAsg304n8=
=OTn+
-----END PGP SIGNATURE-----
martin_raspaud.vcf

bill....@gmail.com

unread,
Apr 19, 2015, 6:04:19 PM4/19/15
to pyt...@googlegroups.com
Hi Ioan

Yes please, we would be interested in seeing that code and how you've made use of header 130.

Thank you very much
Bill

ioan.f...@geomodel.eu

unread,
Apr 21, 2015, 5:25:27 AM4/21/15
to pyt...@googlegroups.com
Bill,


please find attached the code
i use one custom class to calculate lats and lons but you can easily replace that with np.arange, just account for half pixel offset.
I also use gdal and pylab to visualize

 Keep in mind this is work in progress and be suspicious about the code rather than trust it. I tried to write as many comments as I could.

Martin, I am extremely busy with GVAR GOES data processing but as son as I will have some time i will integrate the code into pytroll and let you know about it.

Regards
file.py

ioan.f...@geomodel.eu

unread,
Apr 23, 2015, 2:55:08 AM4/23/15
to pyt...@googlegroups.com, martin....@smhi.se
Martin, 

is the pytroll sprint developer week event fully booked?

Ioan

Martin Raspaud

unread,
Apr 23, 2015, 2:59:08 AM4/23/15
to ioan.f...@geomodel.eu, pyt...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2015-04-23 08:55, ioan.f...@geomodel.eu wrote:
> Martin,
>
> is the pytroll sprint developer week event fully booked?

Hi Ioan,

No it isn't, you're very welcome to join !

Best regards,
Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVOJg7AAoJEBdvyODiyJI4XxAH/A0yzZOY/5jVRWbfrf2BbtNN
BooO9k5Kj8sV+foyp/czDP6PUjpLukmf3n4R2axm3TISofnK2JeAuAVbmCB0cMof
tFrwVkTt8g79M7G7Lb137XtHXDib0UDBOj2CeYMjxK9QPCS8WEp4I3jsNW1QdBiU
1wxHWjHKl59ez1NdJsmgm3tt7Ew9zOmeC4DiW9wQLUZfgAnHM/DIJw/8KMhyTDJR
9dTd0IUXqofGUxLgOun9mBj6FxH27RqmnDcwxowJRGxoZsseMsMysUvtBJGwmoLj
VCbaULxHNEmLwLGHt9g+qaeF7ASBaFLcQsDO0u76vkRQ9JwhJgVwgyEb2p4SNko=
=YVFp
-----END PGP SIGNATURE-----
martin_raspaud.vcf

ioan.f...@geomodel.eu

unread,
Apr 28, 2015, 2:19:07 AM4/28/15
to pyt...@googlegroups.com, martin....@smhi.se, ioan.f...@geomodel.eu
Hello Martin,
I'd like to attend  the  pytroll sprint developer week.
If it's not much of a burden could you send me details so i can book the flights and the hotel? 

Regards

ioan.f...@geomodel.eu

unread,
Jan 14, 2016, 8:42:29 AM1/14/16
to pytroll
Hello Bill,

I'd like to let you know we (GeoModel Solar) are in process of correcting all GMS and MTSAT1R data. We are talking about 6-7 year of data.
GSM5 data is in McIDAS area format featuring GOES navigation except very few files that feature GMS navigation.
MTSAT1R data is in HRIT format.
we have python/numpy based code that can read this data formats

The correction works as follows:

1. read/preprocess the data (also reproject it into GEOS(140) for GMS as almost every image feature a different subsatellite point)
2. using a detailed land sea mask and around 400 sites mostly located in specially selected locations (shore, lakes, etc) compute displacement between the local mask obtaind by classifying the calibrated visible channel in original resolution
into foreground and background and the local land/sea mask
3. interpolate the dx&dy components of vectors into the full space of image obataining dx and dy for every pixel in the image
4. correct he image using nearest neighbor interpolation. We use the displacement info whne reprojeting to latlon to preserve the original data as much as possible

The results are at least decent (average rms 0-2 m )

I wrote you ecause if i recall you said you are interested in this data&correction.

Best
Ioan

bill....@gmail.com

unread,
Jan 25, 2016, 10:03:39 PM1/25/16
to pytroll
Hi Ioan

Yes, I was interested in how you were making use of header 130 of the mtsat2 hrit but what you're doing here is interesting also. Our use of the data is mostly in the moment and the older the data get the less use it has for us.

As such, we have in the end used JMA's own C code to generate netcdf files of himawari8 data and then we have our own python code to re-project that data for various basemaps in to various output formats.

This is working well for us and allows us to add products to the output as we need, relatively quickly.

All the best and thank you for passing on your experiences.
Bill
Reply all
Reply to author
Forward
0 new messages