L2 could not read in log

[jonas...@gmail.com]

Hi

Using FORCE 3.5.1 processing L1>L2 i get the following read error in the log:

/projects/eko/fs2/FORCE/Uganda/L1/T36MWD/S2A_MSIL1C_20180107T075311_N0206_R135_T36MWD_20180107T113513.SAFE: could not read /projects/eko/fs2/FORCE/Uganda/L1/T36MWD/S2A_MSIL1C_20180107T075311_N0206_R135_T36MWD_20180107T113513.SAFE/GRANULE/L1C_T36MWD_A013288_20180107T080754/IMG_DATA/T36MWD_20180107T075311_B01.jp2. could not read /projects/eko/fs2/FORCE/Uganda/L1/T36MWD/S2A_MSIL1C_20180107T075311_N0206_R135_T36MWD_20180107T113513.SAFE/GRANULE/L1C_T36MWD_A013288_20180107T080754/IMG_DATA/T36MWD_20180107T075311_B02.jp2. could not read

Data open OK in QGIS and is labelled DONE in the file-list.

It is more or less 100% clouds but opens ok.

/projects/eko/fs2/FORCE/Uganda/L1/T36MWD/S2A_MSIL1C_20180107T075311_N0206_R135_T36MWD_20180107T113513.SAFE DONE

from *.PRM

# CLOUD DETECTION OPTIONS
# ------------------------------------------------------------------------
MAX_CLOUD_COVER_FRAME = 99
MAX_CLOUD_COVER_TILE  = 99
CLOUD_THRESHOLD  = 0.225
SHADOW_THRESHOLD = 0.02

What can induce the read error and how can I get around it?
Just upgraded to 3.6.1. so I will try that first.

Regards

/Jonas

[jonas...@gmail.com]

Hi again

When re-running the 212 granules receiving the error above using 3.6.1. the result is 143 log files where some was skipped due to cc=100% and six error messages of the type:

/projects/eko/fs2/FORCE/Uganda/L1/T36MWD/S2A_MSIL1C_20181007T075811_N0206_R035_T36MWD_20181007T101505.SAFE: dc:  77.53%. wc:   4.13%. sc:   0.00%. cc:  59.89%. AOD: 0.4129. # of targets: 1/0. Unable to lock file /projects/eko/fs2/FORCE/Uganda/L2//africa/X0045_Y0028/20181007_LEVEL2_SEN2A_BOA.dat (timeout: 599.999371s, nx/ny: 15000/15000). error in cubing Level 2 products
Tiling images failed! Erro ger inometric module.

I.e. some improvement...

Regards

/Jonas

[Stefan]

Not sure about the error in your first post, but from your second post I'm assuming the error did not occur the second time you processed the data?

For the error in the second post:
The process times out after 10mins because no lock on the target file could be acquired. Is it possible that you have several products for T36MWD on the same acquisition date (they would differ in processing baseline and processing date)?
Check if there is another scene with the format: S2A_MSIL1C_20181007T075811_N0XXX_R035_T36MWD_20XXXXXXTXXXXXX.SAFE

If that isn't the case, try to eliminate other reasons that could have caused the filesystem issue.

[david frantz]

Hi Jonas,

also not sure about the 1st issue..

For the 2nd one, it may be that there are some remaining lockfiles from a previous run that was killed/crashed/etc. 

In this case, the lockfiles (*.lock) need to be manually removed such that FORCE can obtain new file locks.

Potentially, it could also mean that the file system is super-busy and a lockfile cannot be written within in a (more or less) reasonable time...

Hope this helps,

David

[jonas...@gmail.com]

Dear David
This error did not repeat, and yes there were some lockfiles, it worked fine after removing these. I think my  processing run out of memory which may caused some strange issues.
/Jonas