Change the "ground" plane from XZ to YZ

[Douglas Holfve]

Hello!
I'm currently in the process of creating a side-scroller game using Unreal Engine 5. I'd like to implement flying enemies and have come to the conclusion that generating a nav-mesh on the YZ-plane would probably be the most reasonable approach (X-axis being my depth). 
I'm however no programmer by trade so my C++ knowledge is limited, although I don't mind taking the time to learn. I've watched a bunch of videos and tried to read up as much as I can when it comes to nav-meshes. 
What I've tried to do now is simply switch the y-axis with X-axis in the Recast Demo  changing the normal angle in rcMarkWalkableTriangles and rcClearUnwalkableTriangles to use norm[0] instead of norm[1]. I've also tried swapping a bunch of the functions bounding box direction from X to Y and vice-versa without any luck. Are there any "easy" tweaks in the recast code I could do to bend it to my needs?

If not I'm starting to think that the best approach would to create my own nav-mesh generation system from scratch using a more grid based approach (mainly to make it easier for me) that I then convert into polygons. I would however very much like to still use all the pathfinding code provided by Detour and I'm quite uncertain one how to proceed in Unreal from this point. I imagine I should create a new C++ class that inherits from RecastNavMesh but from there on out I'm not sure what to look for.

Appreciate any help I can get!

[Graham Pentheny]

Hi!

  Unfortunately there’s some assumptions that are pretty deeply baked into recast.  One of which is the Y-up right-handed (OpenGL style) coordinate system.  Changing that would be a pretty significant amount of work that would touch a lot of the library. 

  The good news is that for your purposes there’s some geometry swizzling you can do to achieve your goal:

1. Take your scene and apply a 90 degree rotation to it before passing it as input to Recast.  The rotation should be such that the input geometry becomes aligned to the XZ plane as ground.

2.  Run Recast navmesh generation on it as normal and save out the navmesh data.  This will give you a navmesh on the XZ plane that is rotated 90 degrees from your input geometry.

3. Simulate the AI navigation using that XZ navmesh, but instead of setting the actor position directly from the result of the AI sim, apply the opposite 90 degree rotation before setting actor positions.  This un-rotates the simulation space so it aligns with the XY plane.  

The idea is to separate the “simulation space” from the “visual space” in which the actors exist and translate between the two.  This will let you use recast and simulate the AI (with detour or something else) in the XZ ground plane, but convert that to XY to actually move the actors.  Hope that helps!

Good luck,

Graham