Re: is is using tile coding or random

[Brian Tanner]

I believe that agent uses this tile coding file:

https://code.google.com/p/bt-agentlib/source/browse/trunk/src/org/rlcommunity/btanner/agentLib/functionApproximators/CMAC/NDCMACTileCoder.java?r=30

It calls the method you are talking about in fillTiles:

private final void fillTiles(int[] tilesToFill, Observation normalizedObservation, long uniqueId) {

//We have our own parameter space.

final int tileStartOffset = 0;

int memorySize = theWeights.length;

final int[] noInts = new int[0];

//Would be nice not to recreate this every time filltiles gets called.

double[] scaledDoubles = new double[normalizedObservation.doubleArray.length];

for (int i = 0; i < scaledDoubles.length; i++) {

scaledDoubles[i] = normalizedObservation.doubleArray[i] * theGridSize;

}

theTC.tiles(tilesToFill, tileStartOffset, numTilings, memorySize, scaledDoubles, noInts);

}

--
Brian Tanner
Operations Manager, Fire Inspector
br...@fireplan.ca
204-975-4901 Extension 102

Fire Plan Strategies: Fire safety training, planning, signs, and supplies across Canada!

On Aug 12, 2015, at 11:17 AM, Amir Hossein Mojarrad <ahmod...@gmail.com> wrote:

Dear Brian 

sorry for disturbing you this much 

i was reading the source code of the "EpsilonGreedyTileCodingSarsaLambda"

in the file : "tilecoader.jave"

we have a class TileCoder with :

public synchronized void tiles ( xxxxxx)

and at the end of the file there are lots of marked source code !!!

i checked the whole files 

i didn't see any file calling TILES mention above 

and i see that the hash were marked commented

// /* hash

// Takes an array of integers and returns the corresponding tile after hashing 

// */ 

the question is where the tilecoader is called in the function ??

i find a C++ file which is very similar to your code. 

https://code.google.com/p/rl-library/source/browse/trunk/projects/agents/sarsa-lambda-tilecoding-cpp/sarsa-lambda-tilecoding-cpp.cpp#46

and i guess that the tile in C++ programm is called in this line :

for (int a=0; a<NUM_ACTIONS; a++)

tiles(&activeFeatures[a][0],NUM_TILINGS,MEMORY_SIZE, inputObservations,NUM_OBSERVATIONS,a);

} but i didn't find where you are calling the tiles? thank you so much

 

--

--

--
Sincerely Yours
Amir Hossein Mojarrad
Enterprise Administrator (MCITP) #SR6485499

ISO 27001- DNV : # 011-THR-IS-00115081
Mikrotik certified  (MTCNA) :# 1111NA006
--------------------------------------------------
IRIK.Co
URL: www.irik.ir

Phone : +98(0)713 6274400

Fax : +98(0)7136274400
Mobile / Viber  : +98(0)917 302 83 29
Email : ahm...@gmail.com
Email : a.moj...@irik.ir
Skype : Amir.Hussein.Modjarrad
MSN : ahmod...@gmail.com

[Brian Tanner]

The hash function and the tiles function are there.  I’m not certain they are identical.  They certainly work though.

--
Brian Tanner
Operations Manager, Fire Inspector
br...@fireplan.ca
204-975-4901 Extension 102

Fire Plan Strategies: Fire safety training, planning, signs, and supplies across Canada!

On Aug 12, 2015, at 11:44 AM, Amir Hossein Mojarrad <ahmod...@gmail.com> wrote:

Brian 

I found the hash function :

 long hash_UNH(int ints[], int num_ints, long m, int increment) { ... }  in TileCoder.java 

is it equal to the codes which you commented at the end of "TileCoder.java"  which are C++ source codes ?

On Wed, Aug 12, 2015 at 9:06 PM, Amir Hossein Mojarrad <ahmod...@gmail.com> wrote:

Dear Brian 

Tank you so much for the reply 

i saw the source code which you mentioned , 

theTC = new TileCoder(); ant etc ....

but the TileCoder.java is all marked as comment

and there are no HASH function or any things for tiling ., 

i guess that my "EpsilonGreedyTileCodingSarsaLambda" is an old version Or something 

i am using  "EpsilonGreedyTileCodingSarsaLambda-Java-R30.tar.gz"

in the attach you can find my TileCoder.java 

i'm wondering where the tiling is taking place ?

thank you so much for your help