Hidden dataset

[Marco Mai]

Hi,

Thanks for organizing the competition. This is my first time in a Kaggle competition, so my question may be a bit dumb. 

My question is about the hidden dataset. How can I test my algorithm if I can't access the dataset? Does it mean that I need to submit the notebook every time to test the code? Is there any sample data that we can access to run a sanity check locally before we submit it? 

Thanks,

Marco

[Eleni Triantafillou]

Hi Marco,

Not at all a dumb question, we are aware that this setup is a little complicated. Thank you for asking!

We recommend starting from this starting notebook which at least alleviates the issue to debug the data loading / other parts and allows you to focus on the unlearning function part.

Generally, there are two paths to 'debugging'. The first is through submitting notebooks, as you said (though the potential caveats here are that the system allows only 5 submissions per day, and the error messages aren't always very helpful). Alternatively (or in addition), you are free to debug offline on any other dataset you would like (the starting kit we released on CIFAR-10 may be helpful for this, for example). Though, of course, if your offline debugging environment (dataset, model, forget set distribution, unlearning metric, etc) is very different from the one we use on kaggle, conclusions may not transfer well. So a good strategy is perhaps a combination of these two paths.

Hope this helps!

Thanks!

--
You received this message because you are subscribed to the Google Groups "unlearning-challenge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unlearning-chall...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unlearning-challenge/614060bc-ed30-4d38-8281-bd5be40148c9n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.