It really sounds like you're running your page with a mobile (iPhone, iPod, iPad, Android, ...) user agent.
In this case, Maps is using touch events to slide map.
What WebInspector says?
Did you try with a previous release? (adding ?v=3.2 in scripts src)
R.