If anyone is interested - I was successfully able to apply the hardware crypto support on top of Neil's excellent 5.15 firmware.
Here's my patch file on top of Neil's gnubee/v5.15 branch of his kernel source.
I took inspiration from this earlier post
Seeing significant increase in crypto benchmarks after this.
Hopefully someone finds this useful.