Race Director will always have limitations with regard to this topic. Gender is a very major part of any race scoring software. Race Director has had a binary gender structure since its inception in 1989. For 30 of the 32 years that I've developed RD, every component of the 250,000 plus lines of code in RD depended on a binary structure. This cannot be changed without a full re-design.
There is a very limited approach now to non-binary in RD, but it's important that you understand the limitations. If you're able to isolate both the registrations and results for the non-binary participants, you can limp by with Race Director. Until you've got a good grasp of what is possible and not possible, I suggest you use Help -> Support well before your race (as Julie has) so that we can look at your setup and advise you on the best possible setup.
The same goes for Race Day Scoring - which is quite a bit more flexible than RD and will have better and better ways of dealing with this topic as time goes on.