ImageNet 2012 is no longer available - a problem for reproducibility

13 views
Skip to first unread message

Grigori Fursin

unread,
Feb 1, 2020, 4:28:16 AM2/1/20
to Collective Knowledge, ctuning-discussions, Artifact Evaluation for systems and ML conferences
Dear colleagues,

Since ImageNet 2012 train and validation sets are not publicly available anymore, the community suggested to print a message about potential workarounds in our CK packages imagenet-2012-val & imagenet-2012-train.

I therefore added a new functionality to our package module to print "problem.txt" from packages and stop execution with an error:

  ck pull repo:ck-env
  ck install package:imagenet-2012-val

  =============================================================================================
  Unfortunately, ImageNet 2012 validation set is no longer publicly available
  and this CK meta-package cannot automatically download it!See the related discussions:

  However, if you already have it installed on your machine, you can detect
  and register it to work with CK workflows using this command:

  $ ck detect soft:dataset.imagenet.val --search_dir={directory where the dataset is installed}
  =============================================================================================

I also published updated packages on our CodeReef platform:

* https://codereef.ai/portal/c/package/imagenet-2012-val
* https://codereef.ai/portal/c/package/imagenet-2012-train

We will keep you informed if we find better ways to handle this situation to ensure the reproducibility of related experiments.

Best,
Grigori

Reply all
Reply to author
Forward
0 new messages