Working with a network stream

[Mahdi Bazargani]

Dear MOA users,

I am very new to MOA. Actually I am working with a large daset which is updated each time and hour from a network so I actually don't know what would be the feature data coming.

I wanted to know how if I can make a stream in MOA..which for example read the data for every 1000 records..and then update all stream using interleaved train and test..

so the stream need to be buffered with new 1000 records and then update the  interleaved train adn test..

What I should do for doing this and how to update  the arrfstream..is there any sample or wiki for this implementation in Java?

Thanks,

Regards,

Mahdi

[Albert Bifet]

You can use EvaluateInterleavedChunks to evaluate batches of data.
Depending on your needs, you may need to create your own stream
reader.

Cheers, Albert

> --
> You received this message because you are subscribed to the Google Groups
> "MOA users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to moa-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Mahdi Bazargani]

Thnaks for the reply,

I an abit confused regrding the diffrences between EvaluateInterleavedChunks and EvaluateInterleavedTrainThenTest

what are the differences between chunckSize and sampleFrequency

I actually suppose EvaluateInterleavedTrainThenTest is a EvaluateInterleavedChunks with chunksize =1 

I guess in EvaluateInterleavedTrainThenTest samples are tested and train one by one but in EvaluateInterleavedChunks samples are tested and tranied by chunks

for example I have quoted these commnets from documentation

===

An example of the EvaluateInterleavedTestThenTrain task creating a comma

separated values file, training the HoeffdingTree classifier on the Waveform-

Generator data, training and testing on a total of 100 million examples, and

testing every one million examples, is the following:

java -cp moa.jar -javaagent:sizeofag.jar moa.DoTask \

"EvaluateInterleavedTestThenTrain -l trees.HoeffdingTree \

-s generators.WaveformGenerator \

-i 100000000 -f 1000000" > htresult.csv

===

What is the meaning of sampleFrequency as they seems test every one million..then what is differences between this and EvaluateInterleavedChunks ?

Thanks for any explanations

[Albert Bifet]

sampleFrequency is only used as the frequency to output the results, not to do any computation.

Cheers, Albert

[Uttam Adhikari]

Dear All, 

I am trying to use MOA to evaluate the Hoeffding trees methods for synchrophasor system (Power system measurement data). I have 3 millions instances of data in ARFF format. I can use the prequential evaluation method on that data and it works fine. however, when I tried to use evaluate interleaved test then train method, it gives me the following error. I am not sure what kind of change do I have to make on the data set or on the MOA setting.

Failure reason: Problems with option: stream

*** STACK TRACE ***java.lang.IllegalArgumentException: Problems with option: stream

                at moa.options.ClassOption.setValueViaCLIString(ClassOption.java:69)

                at moa.tasks.EvaluateInterleavedTestThenTrain.doMainTask(EvaluateInterleavedTestThenTrain.java:103)

                at moa.tasks.MainTask.doTaskImpl(MainTask.java:50)

                at moa.tasks.AbstractTask.doTask(AbstractTask.java:57)

                at moa.tasks.TaskThread.run(TaskThread.java:76)

Caused by: java.lang.Exception: Problem with options to 'ArffFileStream'.

 

Valid options for ArffFileStream:

-f arffFile

ARFF file to load.

-c classIndex (default: -1)

Class index of data. 0 for none or -1 for last attribute in file.

 

                at moa.options.ClassOption.cliStringToObject(ClassOption.java:156)

                at moa.options.ClassOption.setValueViaCLIString(ClassOption.java:67)

                ... 4 more

Caused by: java.lang.IllegalArgumentException: Unknown option: -i

                at moa.options.Options.setViaCLIString(Options.java:149)

                at moa.options.ClassOption.cliStringToObject(ClassOption.java:154)

                ... 5 more

Any suggestion is appreciated. 

Thank you. 

Regards,

Uttam

[Albert Bifet]

>> Failure reason: Problems with option: stream

>> ...
>>IllegalArgumentException: Unknown option: -i

Check your parameter task options. It seems that the problem is in the
parameter -i used in the stream.

Cheers, Albert