Initial Loss x Gain x Entropy

[Weverton Carlos]

I'm trying understand better how Maxent works...

I was analyzing the output files and realized that entropy and gain are related to "Initial Loss", value found in maxent.log:

Initial loss - Gain ≅ entropy of model

 

I think "Initial Loss" is the "initial entropy" of model, before of the computation of gain when the ocurrence points are classified taking into account the Features. Am I right?

If I'm right, how does Maxent calculate the Initial Loss?

Weverton C. Trindade.

State University of Ponta Grossa (Paraná, Brazil)

[Jamie M. Kass]

Just hypothesizing here, as I'm not an expert on the nature of the algorithm itself, but isn't the initial loss the first value of loss calculated by the model before any iterations? From the Maxent help: "The probability is displayed in terms of "gain", which is the log of the number of grid cells minus the log loss (average of the negative log probabilities of the sample locations)."

Jamie Kass
PhD Candidate
City College of NY

[Weverton Carlos]

My doubt was "where does Maxent take that value from?".

I didn't know if the Initial Loss is the entropy of environmental layer values at all landscape, at occurrence points or at background points. I calculated these entropies in R and the Initial Loss corresponds to entropy of environmental layer values at background points.

I'm also not an expert in Machine Learning and algorithms... The information gain is based on the decrease in entropy after a dataset is split on an attribute, right? So, in Maxent, the gain is calculated by:

Gain = entropy after iterarions - entropy before iterations

Is that right? If that is, following the definition of Gain from the Maxent help:

Gain = log of the number of grid cells - log loss (average of the negative log probabilities of the sample locations)

So...

Log of the number of grid cells = entropy before iterations = entropy of background points

Log loss (average of the negative log probabilities of the sample locations) = entropy of model after iterations

Is that right? 

[Jamie M. Kass]

The only accurate way of determing the calculation would be to peek at the code in Java, or the R code forcthe maxnet package. Have you taken a look?

Jamie