Replicating Q-A results with the shared checkpoints

[Reza Esfandiarpoor]

Hi,

I am trying to replicate the r2c results for Q -> A on the validation set with 'best.th' file and the pre-computed Bert embeddings that you shared. My accuracy is 61.9 but the paper reports 63.8!

Do you have any idea about what's going wrong?

I guess it is because of the newer versions of torch and other packages that I use. Can you please share the version of the packages you used to get these results? 

Thanks,

Reza

[Rowan Zellers]

possibly! I have the versions I used on the github README (pytorch 1.0 and an old torchvision branch with RoI pooling) that admittedly are quite different from versions now.

could it also be the case that you reported on val and I reported on test? The sets contain disjoint videos so it's possible there might be some fluctuation between the two.

thanks,

Rowan

[Reza Esfandiarpoor]

Thanks. I check the splits again and also try with this specific version of PyTorch.

Thanks,

Reza