Wow, that works perfectly! Thank you so much!
The --add-gnu-debuglink=x.debug bit was what I was missing.
Unfortunately, it doesn't seem like delve supports separate debug info yet, so I'll have to stick with gdb in these situations.
Also, it seems that upx[1] breaks the debuglink, which doesn't seem entirely unreasonable (I'm mostly including this tidbit in case someone else in a similar situation to mine finds this post)