WORK Download Sentinel 3 Data
May Mcgriff
The Sentinel Operations Center (SOC) coordinates the network of Sentinel Data Partners and leads development of the Sentinel Common Data Model (SCDM), a standard data structure that allows Data Partners to quickly execute distributed programs against local data.
The MSI sensor data are complementary to data acquired by the U.S. Geological Survey (USGS) Landsat 8 Operational Land Imager (OLI) and Landsat 7 Enhanced Thematic Mapper Plus (ETM+) (Comparison of Sentinel-2 and Landsat).
Since 1997, Sentinel has collected tens of millions of consumer reports about fraud, identity theft, and other consumer protection topics. During 2021, Sentinel received over 5.7 million consumer reports, which the FTC has sorted into 29 top categories. The 2021 Consumer Sentinel Network Data Book (Sentinel Data Book) has aggregated information about what consumers told us last year on the full range of fraud, identity theft and other consumer protection topics. The Consumer Sentinel data is also available online in an interactive format at ftc.gov/exploredata, with updates provided quarterly. The Sentinel Data Book is based on unverified reports filed by consumers. The data is not based on a consumer survey. Sentinel has a five-year data retention policy, with reports older than five years purged biannually.
In 2021, the FTC was pleased to welcome the data contributions of the Social Security Administration Office of the Inspector General and the Australian Competition & Consumer Commission. A full listing of data contributors is available in Appendices A3 and A4. Non-government organizations that contribute reports do not have access to Sentinel reports, as access is limited to law enforcement agencies.
The Sentinel-2 mission isa land monitoring constellation of two satellites that provide high resolutionoptical imagery and provide continuity for the current SPOT and Landsat missions.The mission provides a global coverage of the Earth's land surface every 5 days,making the data of great use in on-going studies. L1C data are available fromJune 2015 globally. L2A data are available from November 2016 over Europeregion and globally since January 2017.
I have recently installed SNAP in my Windows8.1 with 8GB RAM and as I said in my first message I am trying to work with SNAP in python using Snappy. I managed to use it properly with the test dataset that is provided with the snappy module in the SNAP directory. I usually work with numpy to handle and process images or multidimensional arrays. But the problem is that the test dataset has really small size if compared with the usual sizes of a Sentinel dataset. And when I try to read for example a Sentinel-2 dataset of small size(1-2 GB) , I manage to read it properly but then if I try to store even only one band in a numpy array I get a MemoryError. So I tried to do the same operation with a smaller subset of the image for a single band and that was successful. So what could I do to work with full images in Python without facing those memory problems? Thanks for the reply!
One is by simply using the SNAP API.
You can find an example in snappy_flh.py. This file is also located in the examples folder in the snappy directory.
In this file you can see that the source data is read line by line. But you can change it to any rectangular shape. This is what I meant by frame. In SNAP terminology frame is in most cases named tile.
The computeTile method is preferred and should be used if results can be computed independently. The computeTileStack should be implemented if the resulting bands are all computed at once, e.g. as output of a neural net.
The computeTile method is also preferable for S2 data because this allows to handle the different resolutions. If the computeTileStack is implemented the source needs to be resampled upfront.
Thank you so much for the links, especially the snappy_flh.py was really helpful, as it shows how to process a sentinel product line by line , and many other things in few lines of code. What I wanted to try also was to display a full image for example with matplotlib but this is probably impossible because of the size, is that the point? anyway your suggestion to consider only subsets of a full image can be the right way. The second approach you described is really interesting but I think it could be useful in a more advanced step of the project I am involved in.
Another question came to my mind while looking at the snappy_flh script: is that possible to create exactly a sentinel-2 product file(I mean with the same structure of an original product, with folders and .xml file) in the same way as it can be done to create for example a geotiff product? The java API is so huge that I do not know where to search for what I need.
Thanks so much so far, for everything
Really thanks for the quick replay. It is really kind from you and I am appreciating it so much.
Actually I did not consider the resampling and in my case it is surely what I need to do.
For all the MERIS sample datasets I had no problem while extracting a subset, and none of them has the multi size issue.
I also noticed another element, in this case it is not an issue, but just about the product structure.
When running the snappy_ndvi.py the product that is returned has the FLAG codings node that is really useful to investigate invidually elements like coastline and water. But the Flag codings node and the correspondant masks are not included in the standard S-2 product if I am not wrong. So in theory it would be possible to do it by ourselves?
In my case I think I could get the flags, but what about creating and setting the corresponding masks?
I have found the mask constructor but I do not understand how to put the data(boolean) into it. I wonder if it could be done with a GPF operator as it has been done to create a band.
Thanks in advance again
Images that contain a great deal of noise have often been processed close to the noise floor (the data closest to the point where it is too noisy to be useful). The noise can look like repeating lines across an image, something like horizontal window blinds, as in the left image below. Those repeating lines are not the same as the bright spots in these images, which appear in the image below and to the right as a line of repeated bright spots or bursts. Those bright bursts are image anomalies that are not yet well understood. Also visible in these images are beam seams (see the next section).
The Precise Orbit Determination (POD) service for Sentinel-1 provides orbit ephemerides files in the form of Precise Orbit Ephemerides files (available 20 days after data acquisition) and Restituted Orbit files (available a few hours after data acquisition). Flight Operation Segment (FOS) Predicted Orbit files (available seven days prior to data acquisition), Instrument Processing Facility (IPF) auxiliary files, L1 Processor Parameters, L2 Processor Parameters, Instrument, Calibration, and Simulated Cross Spectra Auxiliary Data are also available.
1 Regulation (EU) No 377/2014 and Commission Delegated Regulation (EU) No 1159/2013.
2 In agreement with the Copernicus Sentinel Data Policy, ESA/PB-EO(2013)30, rev. 1.
3 See in particular Art. 3 and 9 of Regulation 1159/2013.
4 See in particular Art. 7 of Regulation 1159/2013.
5 See in particular Art. 8 of Regulation 1159/2013.
6 Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data; Regulation (EC) No 45/2001 (EC) No 45/2001 of the European Parliament and of the Council of 18 December 2000 on the protection of individuals with regard to the processing of personal data by the Community institutions and bodies and on the free movement of such data.
The full archive of Sentinel-2 products will be made available by July 2023. Access to Sentinel-2 auxiliary and engineering data will also be made available by November 2023 (access restrictions apply).
The Harmonized Landsat and Sentinel-2 (HLS) project is an extension of research conducted at NASA's Goddard Space Flight Center in Greenbelt, MD, that takes input data from the joint NASA/USGS Landsat 8 and Landsat 9 and the ESA (European Space Agency) Sentinel-2A and Sentinel-2B satellites to generate a harmonized, analysis-ready surface reflectance data product with observations every two to three days.
The HLS project is a major outcome of the Satellite Needs Working Group assessment in 2016. In that assessment, federal agencies and end users identified a need for more frequent Landsat-like observations to track short-term changes in vegetation and other land components to support agricultural monitoring and land cover classification at moderate to high resolution in both the visible and thermal components of the electromagnetic spectrum. Spectral similarities between the Landsat 8 Operational Land Imager (OLI), the Landsat 9 OLI-2, and the Sentinel-2 MultiSpectral Instrument (MSI) present an opportunity to harmonize data from these sensors to generate higher-frequency imagery products for land surface monitoring and applications.
Two data products are generated as part of the HLS project: the L30 data product generated with Landsat 8 and Landsat 9 data, and the S30 product generated using Sentinel-2 data. These data are available through Earthdata Search as well as through NASA's Land Processes Distributed Active Archive Center (LP DAAC). Feedback or questions about HLS data products can be made in the Earthdata Forum for HLS.
HLS data products greatly improve current publicly-available remote sensing land monitoring capabilities, particularly in terms of the frequency of land surface observations through time. The harmonization of HLS ensures that the Landsat 8 and Landsat 9 collection (30-meter spatial resolution with a 16-day repeat period) and the Sentinel-2A/B collection (10 to 20-meter spatial resolution with a five-day repeat period) can be used as if they were a single collection. Through HLS, land surface observations can be acquired at an unprecedented 30-meter spatial resolution every two to three days.
df19127ead