Electro-L N3 (GOMS3) on EUMETCast

[lobsiger...@gmail.com]

Dear Developers

Dear All

EUMETSAT is currently testing dissemination of Electro-L N3 data. This satellite

has replaced Electro-L N2 at 76E (GOMS2 started moving away in summer 2020 ) .

After successful tests the data should be available to the public on Basic Service.

I have access to the test data and made a couple of images with Satpy 0.25.1.

The Satpy 'electrol-hrit.py' reader has the below class for reading Epilogue-Files.

<cite>

class HRITGOMSEpilogueFileHandler(HRITFileHandler):
    """GOMS HRIT format reader."""

    def __init__(self, filename, filename_info, filetype_info):
        """Initialize the reader."""
        super(HRITGOMSEpilogueFileHandler, self).__init__(filename, filename_info,
                                                          filetype_info,
                                                          (goms_hdr_map,
                                                           goms_variable_length_headers,
                                                           goms_text_headers))
        self.epilogue = {}
        self.read_epilogue()

    def read_epilogue(self):
        """Read the prologue metadata."""
        with open(self.filename, "rb") as fp_:
            fp_.seek(self.mda['total_header_length'])
            data = np.fromfile(fp_, dtype=epilogue, count=1)[0]

            self.epilogue.update(recarray2dict(data))


C1 = 1.19104273e-5
C2 = 1.43877523

SPACECRAFTS = {19001: "Electro-L N1",
                                19002: "Electro-L N2"}

</cite>

As you can see at the end is a dict SPACECRAFTS where N3 is missing.

An educated guess would be to add the new satellite N3 as follows:

SPACECRAFTS = {19001: "Electro-L N1",

                                 19002: "Electro-L N2",

                                 19003: "Electro-L N3"}

What is SPACECRAFTS used for? I tried both: Wether I add or don't add the N3 line

I can image N3 data with no special error message seen. What's the right thing to do?

Best regards

Ernst

[Raspaud Martin]

Dear Ernst,

the SPACECRAFTS dictionary is used to set the metadata's
`platform_name` only, so nothing critical for reading.

Really nice that the reader works out of the box for this new
satellite!

Best regards,
Martin

[lobsiger...@gmail.com]

Dear Martin,

thanks for the rapid explanation. In fact EUMETSAT distributed the GOMS3 data the first day without Prologue and Epilogue files.

I could not image the data (error: Prologue/Epilogue might be missing ...) and I informed OPS about the missing files. After your

explanation above I printed out 'satellite_id' in the 'electrol-hrit.py' file handler and it was 19002 and accordingly the platform_name

when I printed out my scene object in the script was 'Electro-L N2'. Does this mean the  satellite_id is wrong in the prologue file?

This might be the case if either EUMETSAT gets the file this way from POCKOSMOC or if EUMETSAT now adds the prologue files

prior to dissemination the way those looked like from GOMS2. Should I contact OPS again and ask to change 19002 to 19003?

**This will give a KeyError if the SPACECRAFTS dict if the reader is not updated with the line 19003: "Electro-L N3" though !?**

Best regards

Ernst

[Raspaud Martin]

Dear Ernst,

At the moment, we set the platform name from that number, so unless we
are using the wrong number (which I don't think we are as we use
`self.prologue['SatelliteStatus']['SatelliteID']`), it looks indeed
like a bug in the data... Better as OPS to fix it, and we will fix
satpy accordingly.

Best regards,
Martin

[Dr. Arnab Muhuri]

PhD Opportunities at Uni Sherbrooke, Canada

Dear All

It will be kind of you if you can share the following opportunity among the students and faculty members of your institute.

I am looking for PhD candidates to undertake studies on microwave remote sensing of soil moisture. We have other possibilities on subjects like the use of satellite gravimetry for groundwater studies and satellite altimetry for continental waters. Backgrounds in physics, remote sensing, or engineering are welcome. The position will be based at Uni Sherbrooke, Canada. The position commences in the fall session, beginning at the end of August 2021.

Interested candidates can write to me at arnab....@alumni.uni-heidelberg.de or arnab.muh...@iitbombay.org.
Looking forward to your support.

Regards
Arnab  

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/d645f115f9c603c8b7ea9c8a00b80139d6f9c442.camel%40smhi.se.

--

Thanks & Regards

Dr. Arnab Muhuri

Gold Medallist for Excellence in PhD Research, IIT Bombay, India

Alexander von Humboldt Fellow, Heidelberg University, Germany

[Raspaud Martin]

Dear Ernst,

Just out of curiosity, did you end up informing OPS about this problem?

Best regards,
Martin

[lobsiger...@gmail.com]

Dear Martin,

Under EUMHD-3711 the problem was explained to OPS 2021/03/26 (see below). I have no answer so far.

Today I found some old decompressed GOMS2 files from 20190408 and from 20200525 in tmp directories

of two unused servers. I was able to image the data again and print out the "satellite_id". I compared with

GOMS3 data from 20210407 and all Prologue files had satellite_id = 19002. So there is no way to distinguish

GOMS2 from GOMS3 data (except for the file naming of course). I still think that this is a bug in the data.

In what reference are these IDs 19001 and 19002 (and maybe 19003) defined? I found the reference given in

the reader but  just saw the "satellite_status" prologue data structure as used in the satpy Electro-L reader.

Best regards,

Ernst

P.S.

I will  contact EUMETSAT OPS again and try to make myself understood ....

----------------------------------------------------------------------------------------------------------------------------------------------------

Re: EUMHD-3711 RE: Trial dissemination of Electro-L N3 MSU-GS data

Von 

Ernst Lobsiger

An

EUMETSAT User Helpdesk

Datum
26.03.2021 14:34

Dear Debbie
Dear OPS

It appears that there is still a bug in the Prologue (?) data files.
The SatPy 'electrol-hrit' reader has used platform identification with:

...
satellite_status = np.dtype([("TagType", "<u4"),
                             ("TagLength", "<u4"),
                             ("SatelliteID", "<u8"),
                             ("SatelliteName", "S256"),
                             ("NominalLongitude", "<f8"),
                             ("SatelliteCondition", "<u4"),
                             ("TimeOffset", "<f8")])

...

SPACECRAFTS = {19001: "Electro-L N1",
               19002: "Electro-L N2"}

...
 satellite_id = self.prologue['SatelliteStatus']['SatelliteID']
 self.platform_name = SPACECRAFTS[satellite_id]
...

When processing Electro-L N3 (GOMS3) files I notice that satellite_id
is set 19002 and platform_name "Electro-L N2" accordingly. So it seems
I have been successful making N3 images in part because the reader found
the old satellite_id 19002 that we have used for Electro-L N2 (GOMS2)
up to last summer. One of the main Satpy developers proposes EUMETSAT
should check the satellite_id for GOMS3 again and the 'electrol-hrit'
reader would then be updated accordingly. Please also check this thread:

https://groups.google.com/g/pytroll/c/2fjnngvdjlI


Best regards
Ernst

[lobsiger...@gmail.com]

Dear Martin,

Dear developers,

in addition to my last post I think I now found the reference that shows that the SatelliteID for GOMS3 should be 19003 as I proposed:

https://www-cdn.eumetsat.int/files/2020-05/pdf_ten_05105_msg_img_data.pdf

See page 83 of 129 of the above file. I have informed EUMETSAT OPS accordingly.

Best regards,

Ernst

[Raspaud Martin]

Dear Ernst,

Thanks a lot for digging this out!
I'll make a quick PR to add 19003 to satpy.

Best regards,
Martin

On Sun, 2021-04-18 at 08:38 -0700, lobsiger...@gmail.com wrote:
> Dear Martin,

> > hrit' reader has used platform identification with:

> >
> > ...
> > satellite_status = np.dtype([("TagType", "<u4"),
> > ("TagLength", "<u4"),
> > ("SatelliteID", "<u8"),
> > ("SatelliteName", "S256"),
> > ("NominalLongitude", "<f8"),
> > ("SatelliteCondition", "<u4"),
> > ("TimeOffset", "<f8")])
> >
> > ...
> >
> > SPACECRAFTS = {19001: "Electro-L N1",
> > 19002: "Electro-L N2"}
> > ...
> > satellite_id = self.prologue['SatelliteStatus']['SatelliteID']
> > self.platform_name = SPACECRAFTS[satellite_id]
> > ...
> >
> > When processing Electro-

> > L N3 (GOMS3) files I notice that satellite_id

> > is set 19002 and platform_name "Electro-

> > L N2" accordingly. So it seems

> > I have been successful making N3 images in part because the reader
> > found
> > the old satellite_id 19002 that we have used for Electro-

> > L N2 (GOMS2)

[Raspaud Martin]

Dear Ernst,

The change has now been integrated in the master branch.

Best regards,
Martin

[lobsiger...@gmail.com]

Dear Martin,

thank you for the update. EUMETSAT wants to start public GOMS3 dissemination soon:

https://www.eumetsat.int/electro-l-n3-data-on-eumetcast

There is a 'Note' that the prologue file still contains 19002 (GOMS2) and Roshydromet

is expected to change that. I don't hold my breath that this will happen anytime soon as

GOMS3 data showed more severe problems with black horizontal and vertical stripes

lately. Sometimes the image data was geographically misaligned and sometimes the

whole 30 min slots were missing. I will report back if these things finally seem fixed.

Best regards,

Ernst