1. The best you could do is to use straight line, non segments.
2. Another problem might be if your ball is not a ball - in Nape's term not a
Circle shape, but
Polygon with many edges). Such bodies will be bouncing even on a straight floor.
3. Also you can try to listen
interaction with vertices and
cancel collisions and do some tweaks (you will need a lot of attempts to get satisfied with this approach ;) )
4. You can disable rotation and move the ball linearly. There still can be issues with sticking on vertices, so again you can try to cancel that collisions.
5. Set ball body type to KINEMATIC and move it by its velocity property.