Expanding on the settings required, note that as you mentioned libhwui is a (shared/dynamically linked) system library. This means it (and the copy of Skia it contains) is bundled with the Android OS on your device, and not something that apps provide. In order to compile and make your own changes to libhwui you would need to build your own copy of Android and flash it to your device. This isn't a typical use case for app developers, and I'm not aware of any official public docs covering all the steps.
All that said, libhwui already exposes the ability to use Vulkan without recompiling anything. This may require a rooted device, but try running `adb shell setprop debug.hwui.renderer skiavk`, and restarting the app.
Since you mention wanting to investigate bottlenecks, I would suggest exploring
tracing with Perfetto if you haven't already. `adb shell setprop debug.hwui.skia_atrace_enabled` should get you some more tracing data from the copy of Skia in libhwui, and you could instrument your own app by adding trace events to your code.