When you're using any interactive types of assets in a group, in Swipe gallery you need to select Disable swipe in the advanced properties dialog.
This will let the user interact.Then use the Gallery Navigation component to move between gallery frames.
I hope this helps!
Thanks
Sukhbir
(Google Web Designer Team)