getting level1 using a shapefile

90 views
Skip to first unread message

Hamidreza Omidvar

unread,
Oct 26, 2020, 2:12:12 PM10/26/20
to FORCE
Hi David,

First of all thank you for such a great package. I really enjoy working on it.

I am trying to download level1 data from the cloud download function "force-level1-csd". 
I am using a polygon shapefile (test.gpkg) as AOI. When using this file, I get to following error:

Searching for footprints / tiles intersecting with geometries of AOI shapefile...

ERROR 1: In ExecuteSQL(): sqlite3_prepare_v2(SELECT landsat.PRFID FROM landsat, test WHERE ST_Intersects(landsat.geom, ST_Transform(test.geom, 4326))):

  no such column: landsat.geom

I was able to download specifying tiles instead of shapefile, but when using shapefile, it does not work.

Should the gpkg have a specific format? I appreciate it if you could help me to solve this problem.

Stefan

unread,
Oct 27, 2020, 5:39:17 AM10/27/20
to FORCE
At this point, force-level1-csd assumes that the layer in your geopackage has the same name as the geopackage itself. If you'd like to use gpkg as input, please make sure that the filename and layer name match

Stefan

Hamidreza Omidvar

unread,
Oct 27, 2020, 5:44:07 AM10/27/20
to FORCE
Thanks Stefan for the response. I had notices this because I got similar error for the name of the layer before so I made sure the layer name and the file name match. However, as you can see from above error, it is giving me an error related to landsat.geom. Do you have any idea on this?

Stefan

unread,
Oct 27, 2020, 6:07:38 AM10/27/20
to FORCE
Could you provide the gpkg and the command you used so I can try to reproduce the issue?
Thanks

Hamidreza Omidvar

unread,
Oct 27, 2020, 6:21:08 AM10/27/20
to FORCE
Here are the command and the test geopackage file:

force-level1-csd -n -c 0,10 -d 20190101,20191230 -s S2B ~/data/force/meta/ ~/data/force/level1/ ~/data/force/level1/l1_pool.txt test.gpkg


test.gpkg.zip

Stefan

unread,
Oct 27, 2020, 6:45:18 AM10/27/20
to FORCE
Thank you. I was not able to reproduce the issue with the file and command from your post.

I suspect there is an issue with the request to the tile server. Try the following and see if you get the Landsat tiles as gpkg output (test_ls_tiles.gpkg)

Stefan

unread,
Oct 27, 2020, 6:47:36 AM10/27/20
to FORCE
Another possibility:

are you running the command from a directory where you don't have writing permissions?

jonas...@gmail.com

unread,
Oct 27, 2020, 6:54:27 AM10/27/20
to FORCE
Works OK for me (converted AOI to shp format) even if  an error message ( ERROR 1: Wrong field type for fid ) are issued.
/Jonas

[jonasardo@aurora-nateko2 EU]$ force-suite force-level1-csd -n -c 0,10 -d 20190101,20191230 -s S2B ../metadata/ ./L1/ ./L1/test.txt  test.shp


Searching for footprints / tiles intersecting with geometries of AOI shapefile...
ERROR 1: Wrong field type for fid

Querying the metadata catalogue for Sentinel2 data
Sensor(s): S2B
Tile(s): T37MDR,T37MDS
Daterange: 20190101 to 20191230
Cloud cover minimum: 0%, maximum: 10%

19 Sentinel2 Level 1 scenes matching criteria found
10.54GB data volume found.

Done.

[jonasardo@aurora-nateko2 EU]$ force-suite force-level1-csd  -c 0,10 -d 20190101,20191230 -s S2B ../metadata/ ./L1/ ./L1/test.txt  test.shp


Searching for footprints / tiles intersecting with geometries of AOI shapefile...
ERROR 1: Wrong field type for fid

Querying the metadata catalogue for Sentinel2 data
Sensor(s): S2B
Tile(s): T37MDR,T37MDS
Daterange: 20190101 to 20191230
Cloud cover minimum: 0%, maximum: 10%

19 Sentinel2 Level 1 scenes matching criteria found
10.54GB data volume found.

Starting to download 19 Sentinel2 Level 1 scenes



Downloading S2B_MSIL1C_20190131T073139_N0207_R049_T37MDR_20190131T094331(4 of 19)...

|=================================================                                  ^C^Z                                                                                                                               |  23 %
[2]+  Stopped                 force-suite force-level1-csd -c 0,10 -d 20190101,20191230 -s S2B ../metadata/ ./L1/ ./L1/test.txt test.shp


Hamidreza Omidvar

unread,
Oct 27, 2020, 7:22:23 AM10/27/20
to FORCE
I was able to use ogr2ogr command to get landsat tiles (attached). I also checked and I have permission to write where I run the command. It is very strange
test_ls_tiles.gpkg.zip

Stefan

unread,
Oct 27, 2020, 8:01:13 AM10/27/20
to FORCE
Very strange, indeed. Is there a file called 'merged.gpkg' in the directory that you ran the command from? If so, does it contain a layer called 'landsat'?

Thanks Jonas for confirming!

Hamidreza Omidvar

unread,
Oct 27, 2020, 8:14:22 AM10/27/20
to FORCE
Yes there are such a file and all the features including landsat, AOI and sentinel are there. I can see the polygons

Stefan

unread,
Oct 27, 2020, 8:37:09 AM10/27/20
to FORCE
I'm sorry but I'm not sure why the operation fails in your case. Only thing I can suggest is to delete the merged.gpkg file and try again.

Hamidreza Omidvar

unread,
Oct 27, 2020, 9:07:50 AM10/27/20
to FORCE
Is the Berlin geopackage shapefile from the tutorial available so I can do a quick test on my system?

Stefan

unread,
Oct 27, 2020, 9:44:36 AM10/27/20
to FORCE
Give it a go, but I'm almost certain that this issue is not related to your AOI gpkg.

Stefan

unread,
Oct 27, 2020, 9:49:41 AM10/27/20
to FORCE
Google Groups doesn't seem to like me attaching the file. You can download it here: https://upload.disroot.org/r/8WS8aDzB#5YLa5ZWAI65JPw8hdRF71WoxSIlGJRnMI0okvUnm0ok=

Hamidreza Omidvar

unread,
Oct 27, 2020, 12:41:27 PM10/27/20
to FORCE
Thank you Stefan. You are right. I managed to run it using docker. There was something wrong with my manual installation that will never know!

Stefan

unread,
Oct 27, 2020, 1:12:41 PM10/27/20
to FORCE
Glad you got it solved! Enjoy fast downloads.

Stefan

unread,
Nov 13, 2020, 7:52:18 AM11/13/20
to FORCE
We were able to track down the error here: https://groups.google.com/g/force_eo/c/lzAOXAEWRHo
If you have the time, it would be great if you could check if the latest dev version of force-level1-csd (https://github.com/ernstste/force/blob/develop/bash/force-level1-csd.sh) fixes the problem for you.
Would also be great to know which version of GDAL and possibly spatialite you're using.
Reply all
Reply to author
Forward
0 new messages