Genetic Algorithm

[Ricardo Martínez Prentice]

Dear  João  , 

I would like to ask about tuning the genetic algorithm to find the fitness function for a OTB segmentation. 

 

I get the following error: 

Error in if (object'at'run >= run) break : 

  missing value where TRUE/FALSE needed

(I am including the 'at' because otherwise, google thinks it is a email adress). At first I thought it was about tuning the parameters for popSize, maxiter and run. After using "low" values I get the same error and it comes from GA:ga as it says as well: 

Error in gaOptimizeSegmentationParams(rstFeatures = rstClassifFeatures,  : 

  An error occurred while running ga function

As stated on your article, I would like to perform the fitness function in order to find the best parameters for segmentation. Minimum parameters are set in a vector of three elements (example): 

OTB_min_params <- c(1, 2, 5)

OTB_max_params <- c(2, 3, 10)

so the code is as follows:

gaOptim <- gaOptimizeSegmentationParams(  rstFeatures = rstClassifFeatures,

trainData   = rstTrainData,

# // Segmentation parameters ---

segmentMethod = "OTB_LSMS",

inputRstPath  = segmFeatures,

pythonPath    = otbPath, ##Perhaps this is the one which fails ... which is the name of the argument?

verbose = FALSE,

# // End segmentation parameters ---

trainThresh   = 0.2,

segmStatsFuns = c("mean","sd"),

classificationMethod = "RF",

classificationMethodParams = NULL,

balanceTrainData = FALSE,

balanceMethod = "ubOver",

evalMethod = "10FCV",

evalMetric = "Kappa",

 minTrainCases = 1,

minCasesByClassTrain = 18,

minCasesByClassTest = 1,

minImgSegm = 1,

lower = OTB_min_params,

upper = OTB_max_params,

popSize = 3, 

pcrossover = 0.9, 

pmutation = 0.1,

maxiter = 20, 

run = 20,

keepBest = TRUE,

parallel = 2) # use two cores for optimization (change to available resources)

Nevertheless, feel free to send me a link for a reference. 

Thank you very much

[João Gonçalves]

Hi Ricardo,

Sorry for the slow response...

This seems to be an issue related to the parameter boundaries defined in OTB_min_params and OTB_max_params. Try to tweak them a little to see if the genetic algorithm starts to pick up good parameter combinations. It may also be related to the RasterStack object defined in rstFeatures input. In any case, without a reproducible example, it is hard to diagnose what is happening.

Also, you are setting very low values in minTrainCases and minImgSegm both equal to 1. This means that you consider admissible having a training dataset with at least one object or any segmentation output that results in 1 object (respectively). The same for  minCasesByClassTest = 1... an evaluation dataset with at least one object/case per class is too low. With these settings, you may end up with non-informative and biased results from SegOptim. Please check them carefully. I guess this may also be an issue related to your train data - have you enough examples for training? Are these examples representative of the objects/targets you want to classify through OBIA?

For sorting this out, turn off the parallel processing (i.e. parallel = FALSE) and see how it goes. Post any error message here for further analysis.

Cheers

João

- - -

[Ricardo Martinez]

Thank you João!

No problem for replies. I am also late. Thank you for your advices.

I solved two errors related to the segmentation parameters. I did a stack for segmFeatures variable, when you do not do it on your tutorial. The other one was the otbPath. 

You are right, my samples are small although quite pure so now I am being careful with those parameters. By visualising the segments in QGIS (raster to polygon) and training dataset, I am trying to use appropriate parameters.

The following error suggests to use another parameter , bylayer,  but I am not sure in which position I should include the argument:

Error in calculateSegmentStats(rstFeatures = rstFeatures, rstSegm = rstSegm,  : 

  Not enough memory to process the input raster files! Modify option bylayer to TRUE

I have set parallel = FALSE  as you recommended. 

Thank you very much!

[João Gonçalves]

Hi Ricardo,

Thanks for your feedback. Seems that you are having some problems fitting your imagery into memory while calculating statistics for each segment (more specifically the rstFeatures raster stack). Unfortunately, this is still an issue in SegOptim which relates to the need for speeding up some calculations in R.

To tackle this there are a couple of solutions but first, you have to make sure you have the experimental build of SegOptim installed (also better to perform a re-installation because of recent changes introduced). To do that run:

remotes::install_github("joaofgoncalves/SegOptim", ref="experimental")

Then, to manage memory issues you may use the following approaches:

    i) (If you are using optimization of image segmentation parameters), try to use a smaller and representative subset area of your overall study site to run the optimization. That way less data is put into memory each time segment statistics are calculated;

    ii) (If you are using optimization of image segmentation parameters), avoid using parallel processing because this will make things faster but consume much more memory (memory usage multiplies by the number of parallel workers, so keep that in mind);

    iii) run segment statistics by layer, to do that, enable these options here:

calculateSegmentStats( <other_options>,  bylayer=TRUE)

Or

gaOptimizeSegmentationParams( <other_options>,  bylayer=TRUE )

    iv) run segment statistics by tiles, which is the best way to reduce memory usage:

calculateSegmentStats( <other_options>,  tiles = 3)

Or

gaOptimizeSegmentationParams( <other_options>,  tiles = 3)

This way your full raster stack in rstFeatures will be break up into smaller rectangular subsets (i.e. tiles) along the x and y axis. In the example above 3 x 3 = 9 tiles would be generated. The number of tiles must be set to avoid an excessively large number of tiles which would make processing very time-consuming.

Hope this helps. Let me know how it goes.

Cheers

João

- - -

[Ricardo Martinez]

Thank you  João . I used the argument "tiles" the gaOptimizeSegmentationParams() function but I get the following error: 

Error in calculateSegmentStats(rstFeatures = rstFeatures, rstSegm = rstSegm,  : 

  The number of tiles across x and y must be an integer number

I guess that as the nrow and col of my raster are not multiple of the square tile, it throws that error (nrow is 8345 and ncol is 6707, so here is the problem, I think). 

Regards,  

[João Gonçalves]

Hi Ricardo,

Please send the script you used for inspection otherwise it is impossible to find and diagnose what is happening. Keep in mind that the tiles parameter is an integer value (N) that will divide your entire scene into a number of blocks equal to N^2

Cheers

[Ricardo Martinez]

Yes of course. 

The segmentation parameters are pointing to raster stacks and the OTB libraries. rstClassifFeatures is a stack of 21 layers. The raster has  8345 rows and 6707 columns. 

Thank you,

# Minimum values for SpectralRange, SpatialRange and Size threshold

OTB_min_params <- c(0.1, 0.2, 5)

# Maximum values for SpectralRange, SpatialRange and Size threshold

OTB_max_params <- c(0.3, 0.4, 20)

gaOptim <- gaOptimizeSegmentationParams(  rstFeatures = rstClassifFeatures, 

                                          trainData   = rstTrainData,

                                          # // Segmentation parameters ---

                                          segmentMethod = "OTB_LSMS",

                                          inputRstPath  = segmFeatures,

                                          otbBinPath    = otbPath, 

                                          verbose = FALSE,

                                          # // End segmentation parameters ---

                                          trainThresh   = 0.05,

                                          segmStatsFuns = c("mean"),

                                          classificationMethod = "RF",

                                          classificationMethodParams = NULL,

                                          balanceTrainData = FALSE,

                                          balanceMethod = "ubOver",

                                          evalMethod = "10FCV",

                                          evalMetric = "Kappa",

                                          trainPerc = 0.7, 

                                          minTrainCases = 30,

                                          minCasesByClassTrain = 10,

                                          minCasesByClassTest = 4,

                                          minImgSegm = 1,

                                          lower = OTB_min_params,

                                          upper = OTB_max_params,

                                          popSize = 10, 

                                          pcrossover = 0.9, 

                                          pmutation = 0.1,

                                          maxiter = 4, 

                                          run = 3, 

                                          keepBest = TRUE,

                                          parallel = FALSE,

                                          tiles = 3)