CHUNK_SIZE ?

[jonas...@gmail.com]

Hi

How and here to to specify CHUNK_SIZE?

Don't find much regarding CHUNK on  https://force-eo.readthedocs.io/en/latest/index.html

I try to process Level2  for LANDSAT 8 with RESOLUTION = 30

I receive in the log file:

/projects/eko/fs4-backup/JA/EU/L8L1/192021/LC08_L1TP_192021_20150318_20170412_01_T1: CHUNK_SIZE must be a multiple of RESOLUTION. If DO_REPROJ = FALSE, the image resolution overrides the RESOLUTION parameter. Starting datacube failed.
Starting datacube(s) failed.

When executing "force-level2" using PRM file:

++PARAM_LEVEL2_START++

# INPUT/OUTPUT DIRECTORIES
# ------------------------------------------------------------------------
FILE_QUEUE = /projects/eko/fs4-backup/JA/EU/L8L1/L8Skane.txt
DIR_LEVEL2 = /projects/eko/fs4-backup/JA/EU/L8L2/
DIR_LOG = /projects/eko/fs4-backup/JA/EU/L8log/
DIR_TEMP = /projects/eko/fs3/tmp

# DIGITAL ELEVATION MODEL
# ------------------------------------------------------------------------
FILE_DEM = /projects/eko/fs3/nateko/bin/Sen2Cor-02.08.00-Linux64/2.8/SRTM/Nordic/Nordic_DEM.vrt
DEM_NODATA = -32767

# DATA CUBES
# ------------------------------------------------------------------------
DO_REPROJ = TRUE
DO_TILE = TRUE
FILE_TILE = NULL
TILE_SIZE = 90000
BLOCK_SIZE = 10000
RESOLUTION_LANDSAT = 30
RESOLUTION_SENTINEL2 = 10
ORIGIN_LON = 10.5
ORIGIN_LAT = 60.5
PROJECTION = PROJCS["SWEREF99 TM",GEOGCS["SWEREF99",DATUM["SWEREF99",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6619"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4619"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",15],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","3006"]]


RESAMPLING = CC

# RADIOMETRIC CORRECTION OPTIONS
# ------------------------------------------------------------------------
DO_ATMO = TRUE
DO_TOPO = TRUE
DO_BRDF = TRUE
ADJACENCY_EFFECT = TRUE
MULTI_SCATTERING = TRUE

# WATER VAPOR CORRECTION OPTIONS - not used for sentinel2
# ------------------------------------------------------------------------
DIR_WVPLUT = /projects/eko/fs4-backup/JA/EU/wvp/global
WATER_VAPOR = NULL

# AEROSOL OPTICAL DEPTH OPTIONS
# ------------------------------------------------------------------------
DO_AOD  = TRUE
DIR_AOD  = NULL

# CLOUD DETECTION OPTIONS
# ------------------------------------------------------------------------
MAX_CLOUD_COVER_FRAME = 90
MAX_CLOUD_COVER_TILE  = 90
CLOUD_THRESHOLD  = 0.225
SHADOW_THRESHOLD = 0.02

# RESOLUTION MERGING
# ------------------------------------------------------------------------
RES_MERGE = NONE

# CO-REGISTRATION OPTIONS
# ------------------------------------------------------------------------
DIR_COREG_BASE = NULL
COREG_BASE_NODATA = -9999

# MISCELLANEOUS OPTIONS
# ------------------------------------------------------------------------
IMPULSE_NOISE = TRUE
BUFFER_NODATA = FALSE

# TIER LEVEL
# ------------------------------------------------------------------------
TIER = 1

# PARALLEL PROCESSING
# ------------------------------------------------------------------------
# Multiprocessing options (NPROC, DELAY) only apply when using the batch
# utility force-level2. They are not used by the core function force-l2ps.
# ------------------------------------------------------------------------
NPROC = 30
NTHREAD = 2
PARALLEL_READS = FALSE
DELAY = 4
TIMEOUT_ZIP = 30


# OUTPUT OPTIONS
# ------------------------------------------------------------------------
OUTPUT_FORMAT = GTiff
OUTPUT_DST = TRUE
OUTPUT_AOD = TRUE
OUTPUT_WVP = TRUE
OUTPUT_VZN = TRUE
OUTPUT_HOT = TRUE
OUTPUT_OVV = TRUE

++PARAM_LEVEL2_END++

Regards

/Jonas

[jonas...@gmail.com]

Hi again

Setting
BLOCK_SIZE = 9000 (instead of 10000)

seem to solve it for resolution = 30

/Jonas

[Stefan]

Hi Jonas,

the block size, just as the tile size, needs to be divisible by the output resolution of the data. Also see here: https://groups.google.com/g/force_eo/c/mhydscNI3XE/m/6pQjbdtHBwAJ

I'm not sure though why it's called chunk size in the log. Maybe David can shed some light on the terminology.

[david frantz]

The terms are used interchangeably!

Best,

David

[david frantz]

I put the homogenization of wording on my todo tab