It depends on what you are trying to do.
If you have a target with an OS, and want to build a cross compiler for that target, then you should extract a sysroot from the target file system. The sysroot needs to include everything the compiler needs, which is basically /usr/include, /lib, /usr/lib, and 64-bit versions of them if they exist. Then use these sysroots to build the cross compiler by specifying --with-sysroot= when configuring.
If you just want a cross compiler, and don't care if the code works on any target as long as it works on a simulator, then you can use
github.com/riscv/riscv-gnu-toolchain to build a cross compiler. This can build either a newlib/elf compiler or a glibc/linux compiler. Unfortunately riscv-gnu-toolchain is temporarily broken as RISC-V International is reorganizing their github trees, but should be working again within a few days.
If you have a target, but no OS, then building a cross compiler is a very complicated procedure that you shouldn't attempt unless you are willing to spend months or years figuring out how this works. It is best to use someone else's scripts, like OpenEmbedded or Gentoo, or just grab a distro like Ubuntu/Fedora/Debian/OpenSuse/etc.
Jim