Evaluating models on leaderboard

41 views
Skip to first unread message

apo...@iiitd.ac.in

unread,
Nov 3, 2019, 1:05:01 AM11/3/19
to Visual Commonsense Reasoning
Is it possible to get test set of VCR? I'm interested in studying the cases where models on leaderboard fail.

rowan....@gmail.com

unread,
Nov 5, 2019, 2:29:06 PM11/5/19
to Visual Commonsense Reasoning
Hi there,

I really like this idea! I was intending to set something like this up (maybe on a limited set of examples), but I never got around to it :( Would it work to get access to the val set?

thanks,
Rowan

apo...@iiitd.ac.in

unread,
Nov 5, 2019, 9:20:28 PM11/5/19
to Visual Commonsense Reasoning
Hey, great. Yes, I will work with val set as test set is unlabelled.
Reply all
Reply to author
Forward
0 new messages