Data Mining with Weka - Lesson 2.6 Activity

[Nick M]

Hi,

I have a problem with the following question of the Lesson 2.6 Activity: With the same dataset, select Percentage split as the test option with 90% as the parameter. Again evaluate J48 with the same seed values: 11, 12, 13, 14, 15
4. What is the mean of the accuracy (in percent; round accuracies to one decimal place)?

Using WEKA version 3.6.12, my results are:

Seed 11	98.7
Seed 12	94.7
Seed 13	98
Seed 14	96
Seed 15	94.7
Average:	96.4

However, the answer suggests that for Seed 12 the accuracy should be 95.3. Am I doing something wrong?

Thanks,

Nick

[Nick M]

I repeated the same steps using 3.6.11 version, and the result is now matching the suggested answer. It appears to be a difference between versions x.11 and x.12 for the seed value 12. Thank you.  

[Ian Witten]

On 27/04/2015, at 10:48 am, Nick M <xyz...@gmail.com> wrote:

I repeated the same steps using 3.6.11 version, and the result is now matching the suggested answer. It appears to be a difference between versions x.11 and x.12 for the seed value 12. Thank you.  

I’d be astonished — and very surprised — if these results differ for different versions of Weka: this is certainly not supposed to happen. Just to repeat the question: with the segment-challenge dataset and percentage split as the test option with 90% as the parameter, evaluate J48 (default options) with these seed values: 11, 12, 13, 14, 15. My answers are 98.7, 95.3, 98.0, 96.0 and 94.7 respectively.

I have confirmed this with several Weka versions, and on several platforms. Yet both Nick and Crawford say they got 94.7 when seed = 12, but the same values as above for the other seeds. 

I’m assuming that this is an error, and my values above are correct and the same for all versions. But if you try it again and still get 94.7 when seed = 12, please could you let me know the version of Weka, the version of Java, and the operating system, and I’ll investigate further.

cheers

ian

On Sunday, April 26, 2015 at 1:15:06 AM UTC-4, Nick M wrote:

Hi,

I have a problem with the following question of the Lesson 2.6 Activity: With the same dataset, select Percentage split as the test option with 90% as the parameter. Again evaluate J48 with the same seed values: 11, 12, 13, 14, 15
4. What is the mean of the accuracy (in percent; round accuracies to one decimal place)?

Using WEKA version 3.6.12, my results are:

Seed 11	98.7
Seed 12	94.7
Seed 13	98
Seed 14	96
Seed 15	94.7
Average:	96.4

However, the answer suggests that for Seed 12 the accuracy should be 95.3. Am I doing something wrong?

Thanks,

Nick

From: Crawford Revie <crawfo...@gmail.com>

Subject: Answer in Lesson 2.6 (Activity 4)

Date: 26 April 2015 10:14:22 am NZST

To: wekamooc...@googlegroups.com

Reply-To: wekamooc...@googlegroups.com

I think that there is an error here, in that the accuracy with the seed = 12 should be 94.7%, not the 95.3% given?

Of course this is a trivial issue, and likely of no significant difference...  except that the point of using pre-defined seeds is to allow for reproducibility, and we get different answers! (when they should be the same)...

[Nick M]

I have updated my old Java Version

7 (update 75) to Version 8 and the results are matching now, thank you!

My computer is running Windows 8.1.

 

[Ian Witten]

Glad you got it sorted out (though I’m still surprised/sceptical that that Java version change made the difference).

cheers

ian

--
You received this message because you are subscribed to the Google Groups "WekaMOOC-general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wekamooc-gener...@googlegroups.com.
To post to this group, send email to wekamooc...@googlegroups.com.
Visit this group at http://groups.google.com/group/wekamooc-general.
To view this discussion on the web, visit https://groups.google.com/d/msgid/wekamooc-general/f72d49e0-ec4c-42ce-bcb7-e5927a4fc779%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Crawford Revie]

And now I have read Nick's post and your response...  I can also confirm that I updated by version of Java (but NOT Weka) between the start and end of the course...  But given the number of variables in the 'Java on Windows' equation, I don't think that I have the energy to go back and try to re-create the 'error'...  One to slot away in the memory bank just in case something similar arises in the future...

Cheers,
Crawford.