Hi,
I would recommend to deform the domain in the usrdat2 subroutine, it will provide a much smoother surface because each GLL point of the mesh will be placed exactly on the wavy surface.
If the mesh is deformed in Gmsh, Nek5000 will interpolate the GLL points using 2nd order reconstruction methods in each element: you can loose a lot of information, especially for low grid resolutions, or high frequency wavy patterns.
I attached 2 helical-domains generated using Gmsh. The first (meshNoGLL) is deformed in Gmsh, the second (meshGLL) in usrdat2. Both have the same grid resolution. It can give you an idea of what you could get with Gmsh.
Best regards,
Bastien