I have a new board here with an ATmega2560 and an ATmega8U2... identical,
for all practical intents and purposes, as far as the microcontrollers are
concerned, to an Arduino Mega 2560.

I have googled around and found the appropriate hex files for both
microcontrollers and programmed them both.

Both 16 MHz crystal oscillators are confirmed to be running.

The ATmega8U2 USB virtual serial port appears in the OS when plugged in.

Pin 13 on the ATmega2560 flashes two times in fast succession with a longer
pause between the double flashes, which I assume is the default pin 13
"test" output from that particular bootloader.

But programming the board via the Arduino environment doesn't work. It
generates one brief blink on the RXD LED and then just sits there repeating
the timeout error.

Binary sketch size: 1,632 bytes (of a 258,048 byte maximum)
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout

This is just with your basic blink sketch, for the sake of example.

If you google this error, lots of people across various forums etc. seem to
report that this is a problem, but nobody has any clear answer about what
causes it or fixes it.

After programming both the same hex files onto the ATmega2560 and the
ATmega8U2 on an Ethermega, the default digital test waveform on pin 13 is
exactly the same, (two high pulses with a short low time between them, and
a longer gap between each pair of pulses) which is what you'd expect, and
programming the board in the normal way via the virtual UART via the
Arduino environment works perfectly fine.

If I cut the UART lines on the board between the ATmega8U2 and the
ATmega2560 and tie them together on the ATmega8U2 side, and test serial
loopback through a serial terminal, then this successfully works, and it
lights up both the ATmega8U2's TXD and RXD LEDs, which is what you'd expect.

Anyone have any idea how to fix it?

Cheers,
  Luke