Caffe models load very slow on Amazon EC2 instances

[Soda Heng]

Hi,

I am trying to test some of the models we have trained onto an amazon ec2 instance (p2.xlarge, p2.8xlarge, p2.16xlarge) and they seem to take forever (>10-15 mins) to load the models. The models load in a couple of seconds on my desktop computer which leads me to believe that amazon is doing some weird virtualization of their GPUs that is causing this tremendous slowdown.

Is anyone else seeing the same results with theirs?  Any help would be very appreciated.