Segmentation Error

[Nathan Thomas]

Hi Pete/Dan,

Im segmenting a Sentinel-2 scene (a selection of bands at/resampled to 10m resolution) but keep coming across the error:

Subsampling the image to read into memory

ERROR 1: Failed to read file: Could not read image data.

ERROR 1: /Users/nmthomas/Documents/Research/America/Louisiana/Sentinel2/QGIS/segs/Lousiana_S2_DOS_Scale_sub_bands_stchd.kea, band 7: IReadBlock failed at X offset 69, Y offset 64

ERROR 1: GetBlockRef failed at X block offset 69, Y block offset 64

libc++abi.dylib: terminating with unexpected exception of type H5::DataTypeIException

Abort trap: 6

Any ideas on what the error could be? Ive tried playing around with the parameters - Im currently using n=20 and k=200

Cheers, Nathan

[Pete Bunting]

Hi Nathan, 

Don’t know is the honest answer. Which function are you running?

For a whole sentinel-2 scene I would recommend using the tiled segmentation function 

rsgislib.segmentation.tiledsegsingle.performTiledSegmentation

Cheers, Pete

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at https://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.

[Nathan Thomas]

Hi Pete,

Im running rsgislibsegmentation.py from my terminal. Its a scene that Ive subset to my study area. Its a 600mb image so its not huge. It gets so far but then falls over. I ran it at home over the weekend and it got as far as eliminating single pixels before stopping.

Nathan

[Pete Bunting]

Sounds strange. Suggest try the tiled version and see whether than goes (result should be pretty much the same) through but otherwise if you could send me the image I could take a look. I have fixed a couple of rounding bugs relating clipping layers to a common image size which might be related to this but those fixes will be in the next realise. 

Cheers, Pete

[Nathan Thomas]

Im trying it with a Landsat scene over the study site and it seems to be running ok, so it must be a problem with the image. The sentinel data comes in 10, 20 and 60m so Ill check that something weird hasn't happened. In the meantime Ill send the data through, should you get chance to take a look.

Nathan

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsub...@googlegroups.com.

[Pete Bunting [pfb]]

Hi Nathan, 

It could be the image - if you want to check gdal_translate to ENVI and then back to KEA (or what ever format you want), ENVI doesn’t support all the meta-data etc so it in-effect ‘cleans’ the file. 

However, I suspect it is more likely a bug in rsgislib where the image extent is defined and connected to the pixel extent and it just so happens that the image you have hits that bug… As I said, I have fixed a bug on this recently so probably best I try it before you spend a load of time on it (other than trying the ENVI thing). 

Cheers, Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

--------------------------------------------------------------------
Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd myfyrwyr.
(Arolwg Cenedlaethol y Myfyrwyr 2016)
www.aber.ac.uk

Top 4 UK university and best in Wales for student satisfaction
(National Student Survey 2016)
www.aber.ac.uk

[Nathan Thomas]

Hi Pete,

Im sending the file over to you now. FYI, I subset it to just R, G, B and NIR (which were all 10m, without the other resampled 20m and 60m bands) and this is running through ok. I guess it may be an extent issue related to the inconsistent pixel sizing/resampling? I subset it using gdalwarp -cutline and I know this can do funny things to the pixel sizing sometimes.

Cheers, Nathan

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsubscribe...@googlegroups.com.

To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at https://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsub...@googlegroups.com.
To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at https://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.

[Pete Bunting [pfb]]

Hi Nathan, 

Have more bands shouldn’t make a difference. That is odd, unless you are running out of memory? 

All the pixels must be the same size etc if the image is stacked. Odd shaped pixels (i.e., difference x and y) could cause problems so might be worth checking. 

Cheers, Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

[Nathan Thomas]

Hi Pete

It could potentially be a memory issue - ill keep an eye on it. Im going back and resampling all of the bands to make sure that they are all at 10m, before I restack them. Ill let you know if it solves anything.

Nathan

 

Nathan M Thomas

CalTech Postdoc at JPL

Radar Science and Engineering

4800 Oak Grove Dr.

300-356A

Pasadena, CA 91109

Phone: +1 818 354 4681

[Pete Bunting [pfb]]

Hi Nathan, 

I ran through the image you sent and it went through without any problems as it was using all the bands. I ran it through the latest version of rsgislib but I haven’t had chance to run it through the last stable release to see if I can reproduce the error to check whether it is fixed in the work I’ve done since that release. I’ll try that tonight and let you know. The next release is planned for December. 

If you wanted to try the latest dev version yourself it is in conda within the au-eoed channel but bear in mind that I delete and move versions around in there for my own testing and change it adhoc’ly and sometimes testing version up there are broken in someway so I wouldn’t recommend using as your main version. If you do download create a new environment. 

conda install -c conda-forge -c au-eoed rsgislib

Cheers, Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

[Nathan Thomas]

Hi Pete,

Thanks. Ive managed to run it through here too, now that I was more careful with my resampling and subsetting. I should be able to work with this for now and then clip it to the exact extent post-classification. I will create a new environment with the development version for now so at least I have it, should I need it before the release date.

Cheers, Nathan

Nathan M Thomas

CalTech Postdoc at JPL

Radar Science and Engineering

4800 Oak Grove Dr.

300-356A

Pasadena, CA 91109

Phone: +1 818 354 4681

[Pete Bunting [pfb]]

Hi Nathan, 

Just to let you know, I haven’t been able to reproduce the error you had in the current release of rsgislib. Don’t know what’s gone on there. Anyway, if you have it running it probably doesn’t matter for now. 

Cheers, Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

[Nathan Thomas]

Hi Pete,

Hmm, it is a bit weird. Thanks for taking a look for me. I have it working for now at least.

Cheers, Nathan

Nathan M Thomas

CalTech Postdoc at JPL

Radar Science and Engineering

4800 Oak Grove Dr.

300-356A

Pasadena, CA 91109

Phone: +1 818 354 4681