Raspbian is just a modified version of Debian, which (in this context) runs
the Linux kernel on an ARM core. So any book that covers Linux system
programming should do.
The main differences from PCs are that ARM hardware is slightly different to
Intel, so things can be structured in a different way (less discoverable
PCIe, more devices that need to be listed in the Device Tree). And also the
Pi has somewhat peculiar 'firmware' (which lives on the GPU) that's a bit
different to most ARM platforms. But in general Linux tries to smooth over
hardware differences so once you're away from drivers, compilers and
managing the MMU there isn't so much difference between ARM and x86 Linux.
This book:
https://www.wiley.com/en-gb/Learning+Computer+Architecture+with+Raspberry+Pi-p-9781119183938
covers the hardware.
I haven't seen the inside of this book:
https://www.arm.com/resources/education/books/operating-systems
but it looks promising.
Otherwise look at books for 'Linux system programming', 'Linux internals',
'Linux kernel', etc. 95% of them will apply to the Pi.
Theo