Building Xvnc in TigerVNC isn't particularly easy, and I'm speaking as
the guy who used to generate the official binaries for this project
"back in the day." The most common way of doing it-- the method that
O/S distributors use-- involves building the server against pre-built
X11 SDKs and X.org source that match the running X server on the machine
on which you are building. But of course that produces a server that is
specific to that machine. Otherwise, to produce a cross-compatible
server, you have to build all of the X.org infrastructure from source
and then build the TigerVNC Server on top of it. I used to use the
build-xorg script in order to produce cross-compatible builds "back in
the day", but it's horribly outdated. I think Brian is now producing
cross-compatible builds using his own RPM and dpkg scripts that are
specific to a particular Red Hat or Ubuntu distribution (refer to
contrib/ in the source.) He would be the best person to tell you
whether the stuff under contrib/packages/deb will work on an ARM Ubuntu
distribution.
I should also point out, because we're one big VNC family here, that the
TurboVNC Server also supports XKEYBOARD and has a fully-contained Xvnc
source base. So if you run into a snag building the TigerVNC Server,
you may have better luck with ours, since it does do what you expect (it
builds Xvnc whenever you type "make".) The two servers should be mostly
compatible.