Electro-L N3 (GOMS3) on EUMETCast

60 views
Skip to first unread message

lobsiger...@gmail.com

unread,
Mar 25, 2021, 6:54:55 PMMar 25
to pytroll
Dear Developers
Dear All

EUMETSAT is currently testing dissemination of Electro-L N3 data. This satellite
has replaced Electro-L N2 at 76E (GOMS2 started moving away in summer 2020 ) .
After successful tests the data should be available to the public on Basic Service.

I have access to the test data and made a couple of images with Satpy 0.25.1.
The Satpy 'electrol-hrit.py' reader has the below class for reading Epilogue-Files.

<cite>
class HRITGOMSEpilogueFileHandler(HRITFileHandler):
    """GOMS HRIT format reader."""

    def __init__(self, filename, filename_info, filetype_info):
        """Initialize the reader."""
        super(HRITGOMSEpilogueFileHandler, self).__init__(filename, filename_info,
                                                          filetype_info,
                                                          (goms_hdr_map,
                                                           goms_variable_length_headers,
                                                           goms_text_headers))
        self.epilogue = {}
        self.read_epilogue()

    def read_epilogue(self):
        """Read the prologue metadata."""
        with open(self.filename, "rb") as fp_:
            fp_.seek(self.mda['total_header_length'])
            data = np.fromfile(fp_, dtype=epilogue, count=1)[0]

            self.epilogue.update(recarray2dict(data))


C1 = 1.19104273e-5
C2 = 1.43877523

SPACECRAFTS = {19001: "Electro-L N1",
                                19002: "Electro-L N2"}

</cite>

As you can see at the end is a dict SPACECRAFTS where N3 is missing.
An educated guess would be to add the new satellite N3 as follows:

SPACECRAFTS = {19001: "Electro-L N1",
                                 19002: "Electro-L N2",
                                 19003: "Electro-L N3"}

What is SPACECRAFTS used for? I tried both: Wether I add or don't add the N3 line
I can image N3 data with no special error message seen. What's the right thing to do?

Best regards
Ernst


Electro-L-20210325-SLO-1330-ir_overview-full_scan.jpg

Raspaud Martin

unread,
Mar 26, 2021, 4:42:12 AMMar 26
to pyt...@googlegroups.com
Dear Ernst,

the SPACECRAFTS dictionary is used to set the metadata's
`platform_name` only, so nothing critical for reading.

Really nice that the reader works out of the box for this new
satellite!

Best regards,
Martin

lobsiger...@gmail.com

unread,
Mar 26, 2021, 6:40:42 AMMar 26
to pytroll
Dear Martin,

thanks for the rapid explanation. In fact EUMETSAT distributed the GOMS3 data the first day without Prologue and Epilogue files.
I could not image the data (error: Prologue/Epilogue might be missing ...) and I informed OPS about the missing files. After your
explanation above I printed out 'satellite_id' in the 'electrol-hrit.py' file handler and it was 19002 and accordingly the platform_name
when I printed out my scene object in the script was 'Electro-L N2'. Does this mean the  satellite_id is wrong in the prologue file?
This might be the case if either EUMETSAT gets the file this way from POCKOSMOC or if EUMETSAT now adds the prologue files
prior to dissemination the way those looked like from GOMS2. Should I contact OPS again and ask to change 19002 to 19003?
**This will give a KeyError if the SPACECRAFTS dict if the reader is not updated with the line 19003: "Electro-L N3" though !?**

Best regards
Ernst

Raspaud Martin

unread,
Mar 26, 2021, 8:22:53 AMMar 26
to pyt...@googlegroups.com
Dear Ernst,

At the moment, we set the platform name from that number, so unless we
are using the wrong number (which I don't think we are as we use
`self.prologue['SatelliteStatus']['SatelliteID']`), it looks indeed
like a bug in the data... Better as OPS to fix it, and we will fix
satpy accordingly.

Best regards,
Martin

Dr. Arnab Muhuri

unread,
Mar 26, 2021, 4:45:59 PMMar 26
to pyt...@googlegroups.com
PhD Opportunities at Uni Sherbrooke, Canada

Dear All

It will be kind of you if you can share the following opportunity among the students and faculty members of your institute.

I am looking for PhD candidates to undertake studies on microwave remote sensing of soil moisture. We have other possibilities on subjects like the use of satellite gravimetry for groundwater studies and satellite altimetry for continental waters. Backgrounds in physics, remote sensing, or engineering are welcome. The position will be based at Uni Sherbrooke, Canada. The position commences in the fall session, beginning at the end of August 2021.

Interested candidates can write to me at arnab....@alumni.uni-heidelberg.de or arnab.muh...@iitbombay.org.
Looking forward to your support.

Regards
Arnab  

--
You received this message because you are subscribed to the Google Groups "pytroll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytroll+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pytroll/d645f115f9c603c8b7ea9c8a00b80139d6f9c442.camel%40smhi.se.


--
Thanks & Regards
Dr. Arnab Muhuri
Gold Medallist for Excellence in PhD Research, IIT Bombay, India
Alexander von Humboldt Fellow, Heidelberg University, Germany

Raspaud Martin

unread,
Apr 12, 2021, 7:23:28 AMApr 12
to pyt...@googlegroups.com
Dear Ernst,

Just out of curiosity, did you end up informing OPS about this problem?

Best regards,
Martin

lobsiger...@gmail.com

unread,
Apr 17, 2021, 3:14:53 PMApr 17
to pytroll
Dear Martin,

Under EUMHD-3711 the problem was explained to OPS 2021/03/26 (see below). I have no answer so far.
Today I found some old decompressed GOMS2 files from 20190408 and from 20200525 in tmp directories
of two unused servers. I was able to image the data again and print out the "satellite_id". I compared with
GOMS3 data from 20210407 and all Prologue files had satellite_id = 19002. So there is no way to distinguish
GOMS2 from GOMS3 data (except for the file naming of course). I still think that this is a bug in the data.

In what reference are these IDs 19001 and 19002 (and maybe 19003) defined? I found the reference given in
the reader but  just saw the "satellite_status" prologue data structure as used in the satpy Electro-L reader.

Best regards,
Ernst

P.S.
I will  contact EUMETSAT OPS again and try to make myself understood ....


----------------------------------------------------------------------------------------------------------------------------------------------------
Re: EUMHD-3711 RE: Trial dissemination of Electro-L N3 MSU-GS data

Von
 
Ernst Lobsiger
An
EUMETSAT User Helpdesk
Datum
26.03.2021 14:34


Dear Debbie
Dear OPS

It appears that there is still a bug in the Prologue (?) data files.
The SatPy 'electrol-hrit' reader has used platform identification with:

...
satellite_status = np.dtype([("TagType", "<u4"),
                             ("TagLength", "<u4"),
                             ("SatelliteID", "<u8"),
                             ("SatelliteName", "S256"),
                             ("NominalLongitude", "<f8"),
                             ("SatelliteCondition", "<u4"),
                             ("TimeOffset", "<f8")])

...

SPACECRAFTS = {19001: "Electro-L N1",
               19002: "Electro-L N2"}
...
 satellite_id = self.prologue['SatelliteStatus']['SatelliteID']
 self.platform_name = SPACECRAFTS[satellite_id]
...

When processing Electro-L N3 (GOMS3) files I notice that satellite_id
is set 19002 and platform_name "Electro-L N2" accordingly. So it seems
I have been successful making N3 images in part because the reader found
the old satellite_id 19002 that we have used for Electro-L N2 (GOMS2)
up to last summer. One of the main Satpy developers proposes EUMETSAT
should check the satellite_id for GOMS3 again and the 'electrol-hrit'
reader would then be updated accordingly. Please also check this thread:

https://groups.google.com/g/pytroll/c/2fjnngvdjlI


Best regards
Ernst

GOMS-Prologue.PNG

lobsiger...@gmail.com

unread,
Apr 18, 2021, 11:38:28 AMApr 18
to pytroll
Dear Martin,
Dear developers,

in addition to my last post I think I now found the reference that shows that the SatelliteID for GOMS3 should be 19003 as I proposed:


See page 83 of 129 of the above file. I have informed EUMETSAT OPS accordingly.


Best regards,

Ernst

Raspaud Martin

unread,
Apr 19, 2021, 2:17:43 AMApr 19
to pyt...@googlegroups.com
Dear Ernst,

Thanks a lot for digging this out!
I'll make a quick PR to add 19003 to satpy.

Best regards,
Martin

On Sun, 2021-04-18 at 08:38 -0700, lobsiger...@gmail.com wrote:
> Dear Martin,
> > hrit' reader has used platform identification with:
> >
> > ...
> > satellite_status = np.dtype([("TagType", "<u4"),
> > ("TagLength", "<u4"),
> > ("SatelliteID", "<u8"),
> > ("SatelliteName", "S256"),
> > ("NominalLongitude", "<f8"),
> > ("SatelliteCondition", "<u4"),
> > ("TimeOffset", "<f8")])
> >
> > ...
> >
> > SPACECRAFTS = {19001: "Electro-L N1",
> > 19002: "Electro-L N2"}
> > ...
> > satellite_id = self.prologue['SatelliteStatus']['SatelliteID']
> > self.platform_name = SPACECRAFTS[satellite_id]
> > ...
> >
> > When processing Electro-
> > L N3 (GOMS3) files I notice that satellite_id
> > is set 19002 and platform_name "Electro-
> > L N2" accordingly. So it seems
> > I have been successful making N3 images in part because the reader
> > found
> > the old satellite_id 19002 that we have used for Electro-
> > L N2 (GOMS2)

Raspaud Martin

unread,
Apr 20, 2021, 2:50:49 AMApr 20
to pyt...@googlegroups.com
Dear Ernst,

The change has now been integrated in the master branch.

Best regards,
Martin

lobsiger...@gmail.com

unread,
Apr 25, 2021, 7:08:42 AMApr 25
to pytroll
Dear Martin,

thank you for the update. EUMETSAT wants to start public GOMS3 dissemination soon:


There is a 'Note' that the prologue file still contains 19002 (GOMS2) and Roshydromet
is expected to change that. I don't hold my breath that this will happen anytime soon as
GOMS3 data showed more severe problems with black horizontal and vertical stripes
lately. Sometimes the image data was geographically misaligned and sometimes the
whole 30 min slots were missing. I will report back if these things finally seem fixed.

Best regards,
Ernst
Reply all
Reply to author
Forward
0 new messages