ImageNet 2012 is no longer available - a problem for reproducibility

Skip to first unread message

Grigori Fursin

Feb 1, 2020, 4:28:17 AM2/1/20
to Collective Knowledge, ctuning-discussions, Artifact Evaluation for systems and ML conferences
Dear colleagues,

Since ImageNet 2012 train and validation sets are not publicly available anymore, the community suggested to print a message about potential workarounds in our CK packages imagenet-2012-val & imagenet-2012-train.

I therefore added a new functionality to our package module to print "problem.txt" from packages and stop execution with an error:

  ck pull repo:ck-env
  ck install package:imagenet-2012-val

  Unfortunately, ImageNet 2012 validation set is no longer publicly available
  and this CK meta-package cannot automatically download it!See the related discussions:

  However, if you already have it installed on your machine, you can detect
  and register it to work with CK workflows using this command:

  $ ck detect soft:dataset.imagenet.val --search_dir={directory where the dataset is installed}

I also published updated packages on our CodeReef platform:


We will keep you informed if we find better ways to handle this situation to ensure the reproducibility of related experiments.


Reply all
Reply to author
0 new messages