RSGISLib kills kernel, no message why

[JPodgo]

Hello!

I am trying to classify Sentinel 2 images with RSGISLib following this tutorial:  https://spectraldifferences.wordpress.com/2014/09/07/object-based-classification-using-random-forests/

I downloaded a section of image with my ROI from Google Earth Engine as .tif image and converted it to .kea with python gdal.translate() command.

Then I successfully segmented it with shepherd segmentation and prepared a .shp with training polygons, which I rasterized to KEA with gdal_rasterize. All my .kea products open correctly in TuiView.

Problem arises when I try to call populateRATwithStats function. The Python kernel just simply gives me a prompt, that it died and restarts. I don't get any more info when I run a script in Spyder and in Anaconda prompt when the code run as Jupyter notebook.

I am on Windows 10, I make sure to run Conda as administrator, have it installed for all users and the problem remains regardless of whether I work in C or D drive.

How do I make sure all RSGISLib functions work correctly?

Regards

Julian

The code below:

# Hand-coded paths to images for testing
multibandPathTif = os.path.join(sentinelDownloadFolder,'S2_L1_20190203.tif')
multibandPath = os.path.join(sentinelDownloadFolder,'S2_L1_20190203.kea')
clumpsImagePath = os.path.join(segmentationsFolder,'S2_L1_20190203_segm.kea')

# Convert multiband to KEA
tifInput = gdal.Open(multibandPathTif)
formatName = 'KEA'
driver = gdal.GetDriverByName(formatName)
multibandKeaDS = driver.CreateCopy(multibandPath,tifInput,0)
del tifInput
del multibandKeaDS
print('Image converted to KEA')

#==# PREPARE FOR CLASSIFICATION: INPUTS ARE MULTIBAND PATH, .KEA WITH SEGMENTS AND .KEA WITH TRAINING DATA

bandNames = ['Blue','Green','Red','NIR']
trainingPolysFile = r'S2_L1_20190203_classTraining.kea'
classImagePath = os.path.join(trainingFolder,trainingPolysFile)

# Populate with all statistics (min, max, mean, standard deviation)
# Build statistics
bandinfo = []
numBands = len(bandNames) # As many as there are bands
for i in range(numBands):
    bandName = bandNames[i]
    bandinfo.append(rsgislib.rastergis.BandAttStats(band=i, minField=bandName+'Min', maxField=bandName+'Max', meanField=bandName+'Mean', stdDevField=bandName+'Stdev'))
print('Variables for training defined')

# Populate for each clump
rsgislib.rastergis.populateRATWithStats(multibandPath, clumpsImagePath, bandinfo) # HERE IS WHERE THE KERNEL DIES
print('Clumps populated with variables')

[Peter Bunting]

Hi Julian, 

The error is most likely to be due to the band numbering which uses the GDAL convention of starting a 1 rather than 0. You need to change the following line to:

bandinfo.append(rsgislib.rastergis.BandAttStats(band=i+1

I’ll be honest, I am not very good at maintaining them and they don’t test everything in the library but there some tests you can run to check things are working OK. They can be downloaded from the project git repo:

https://github.com/remotesensinginfo/rsgislib/tree/master/python_tests/RSGISLibTests

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK

*
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/23da6be1-cc61-4874-a5a8-cfae46920a49o%40googlegroups.com.

[JPodgo]

Hi Pete,

Thank you for the quick answer! For now I tried to change the band numbering and all I've gotten is a more concrete error message, at least it's something:

ERROR 1: Failed to read attributes: H5Dread failed

I'm however not able to tell which file is the culprit.

I'll try to run the tests, I didn't know about them!

W dniu czwartek, 11 czerwca 2020 16:15:55 UTC-4 użytkownik Peter Bunting napisał:

Hi Julian, 

The error is most likely to be due to the band numbering which uses the GDAL convention of starting a 1 rather than 0. You need to change the following line to:

bandinfo.append(rsgislib.rastergis.BandAttStats(band=i+1

I’ll be honest, I am not very good at maintaining them and they don’t test everything in the library but there some tests you can run to check things are working OK. They can be downloaded from the project git repo:

https://github.com/remotesensinginfo/rsgislib/tree/master/python_tests/RSGISLibTests

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK

*
* Email...@aber.ac.uk

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib...@googlegroups.com.

[Peter Bunting]

Only the segmentation needs to be a KEA file the multi band and training data images can be any format your GDAL installation supports so you could leave them as GTIFF which would help eliminate files. 

Secondly, I don’t know how up to date the spectral difference post is as something might have changed since 2014. I have recent tutorials here:

This one has an example of a segmentation classification and pixel base machine learning classification:

https://www.remotesensing.info/introduction-to-rsgislib-tutorial.html

This one has examples of per-pixel base classifiers:

https://www.remotesensing.info/rsgislib-introduction-to-classification-tutorial.html

The per-pixel based classifier functions were refactored in the most recent release so they tutorials will be the only ones up to date with respect to those.

Cheers, Pete

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/8ed7f49d-fa6d-438d-b679-6206befd8b5fo%40googlegroups.com.

[JPodgo]

Thanks for the new links. I was drawn to RSGISLib for its Object-based analysis potential, but will look at the new tutorials once I get everything running.

I ran the tests and it seems all is fine, at least everything relevant for my raster analyses:

84 TESTS COMPLETED - 1 FAILURES LISTED BELOW:
FAIL: testBufferVector

But I managed to hit a "kernel restart" error with the tester script when I passed an erroneous path to a file (essentially missed a slash between folder names), so maybe that's a clue.

[JPodgo]

Hello Pete!

I seem to generate problems faster than solutions... I somehow worked around the first error by changing the band numbering as you suggested and manually setting environmental variable for PROJ library (which turned out to be the deep reason for some of the crashes). Nevertheless I moved my attempts to create a training dataset to the ratutils.populateClumpsWithClassTraining function, after other vectorisation functions (copyVec2RAT and copyShapefile2RAT) simply did not work (a raster image was created with correct map of polygons, but then kernel died and the .shp attributes were never written to the RAT). Then I got the following message:

RuntimeError: The HDF5ImageDataset driver does not support update access to existing datasets.

Is it possible, that GDAL was recently "updated" to remove updating priviledges and this crippled the RSGISLib tools?

[Daniel Clewley]

Hi,

The message from the tests means none have been run, if you run again with the —-all flag this will run all the tests or you can specify the tests to run with different flags, pass in -h to see them all.

RSGISLib hasn’t been tested much under Windows so you might have more luck using the Linux version through the windows Linux subsystem.

Thanks,

Dan

On 12 Jun 2020, at 01:04, JPodgo <paaj...@gmail.com> wrote:



Thanks for the new links. I was drawn to RSGISLib for its Object-based analysis potential, but will look at the new tutorials once I get everything running.

For now the testing scripts seem to not run at all, does the following output mean all is fine?

COPYING DATA
Removing test files
0 TESTS COMPLETED - 0 FAILURES LISTED BELOW:

--

You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/462178b4-76d8-4646-93ca-1ae0f064970eo%40googlegroups.com.

[Daniel Clewley]

Hi,

This looks like a problem with the KEA driver in GDAL not being recognised so it is trying to open as a  DF5 file instead.

Dan

On 14 Jun 2020, at 00:38, JPodgo <paaj...@gmail.com> wrote:



--

You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/a2d19bd5-cbd7-4a8c-8225-2e931a72e401o%40googlegroups.com.

[JPodgo]

Hi! I managed to jump over all the hurdles and set up RSGISLib in Colab environment. I have described the process in the answer to this question on StackExchange:

https://gis.stackexchange.com/questions/364421/how-to-make-proj-work-via-anaconda-in-google-colab/370360#370360

In short you need to manually set an environment variable for the PROJ library for it to be recognised and remove Colab's built-in OSGeo to make sure only the one that comes with RSGISLib (and KEA driver) is used.