thunderbolt-to-thunderbolt

[Andy Burns]

I remember way back, I had two machines with firewire ports, you could
plug a passive lead between them and each would see an 800Mbps NIC, this
was years before any machine I owned had a 1Gbps ethernet port.

So now if you plug a USB4 cable between two machines with TB4 ports, why
can't they both show a 40Gbps NIC?

[Theo]

You could do that with TB2 - it's how Apple did their machine migration.
Plug in a TB2 cable and network interfaces would appear at each end, and
mDNS took care of addressing and discovery.

I think that migration method still works on TB3/4, but don't have such a
machine to try it.

Maybe you need a specific TB4 not USB4 cable?

Theo

[Andy Burns]

Theo wrote:

> Andy Burns wrote:
>
>> if you plug a USB4 cable between two machines with TB4 ports, why
>> can't they both show a 40Gbps NIC?
>

> Maybe you need a specific TB4 not USB4 cable?

Actually it is a USB4/TB4 cable that came with a CalDigit dock.

[Jaimie Vandenbergh]

On 28 Dec 2023 at 19:24:34 GMT, "Andy Burns" <use...@andyburns.uk>
wrote:

Works at 20Gbps between two Macs with TB3 ports, with full Thunderbolt
cable.

Perhaps you need to create the interface? Go to Settings/Network and
Add, Thunderbolt Bridge. When you plug in the Thunderbolt cable that may
also give you a Thunderbolt Ethernet (Slot 0 or 1 in my case).

If it doesn't, Add again and see if it's in there.

Cheers - Jaimie
--
"Meanwhile, guinea pigs are displaying the survival
instincts of lemmings ... quite astonishingly, 2.86 per
cent of the little blighters have been damaged by a
karaoke machine."
http://www.theregister.co.uk/2008/10/17/pet_wii_problem/

[Andy Burns]

Jaimie Vandenbergh wrote:

> Andy Burns wrote:
>
>> if you plug a USB4 cable between two machines with TB4 ports, why
>> can't they both show a 40Gbps NIC?
>
> Works at 20Gbps between two Macs with TB3 ports, with full Thunderbolt
> cable.

I've seen videos of it being demoed mac->pc with talk of pc->pc, and I
have no doubt it works mac->mac

> Perhaps you need to create the interface? Go to Settings/Network and
> Add, Thunderbolt Bridge.

I think the thunderbolt bridge is a mac-only thing.

> When you plug in the Thunderbolt cable that may
> also give you a Thunderbolt Ethernet (Slot 0 or 1 in my case).

No additional NIC devices appear in Device Manager when connecting the
two laptops via TB4 cable, no extra USB4 endpoints appear on the

> If it doesn't, Add again and see if it's in there.

both laptops have two thunderbolt ports, normally one laptop is
connected to a TB4 dock, and the other port unused

the second laptop normally is just powered over one port and the other
used for occasional USB devices (though it does work as thunderbolt if
plugged to the same dock as above)

have tried undocking laptop so the only ports in use are between the two
laptops with a cable.

Discovered there is a utility called "Thunderbolt Control Centre" in the
Windows store, which has some sort of role in authenticating devices
(presumably to prevent the remote-DMA type attacks from firewire days?)
it shows both ports, but no devices that it can authenticate.

I think I've got a spare thunderbolt downstream port on the dock, might
try the second laptop into that, but seems a bit unlikely it'd work that
way when direct port->port doesn't.

Very little seems to be talked about thunderbolt networking, apart from
physical ethernet dongles, by the laptop manufacturers or microsoft :-(

[Theo]

Andy Burns <use...@andyburns.uk> wrote:
> Jaimie Vandenbergh wrote:
>
> > Andy Burns wrote:
> >
> >> if you plug a USB4 cable between two machines with TB4 ports, why
> >> can't they both show a 40Gbps NIC?
> >
> > Works at 20Gbps between two Macs with TB3 ports, with full Thunderbolt
> > cable.
>
> I've seen videos of it being demoed mac->pc with talk of pc->pc, and I
> have no doubt it works mac->mac

It does exist pc->pc, or at least did.
https://www.reddit.com/r/Thunderbolt/comments/yae2bv/ethernetoverthunderbolt_connection_between/

> > Perhaps you need to create the interface? Go to Settings/Network and
> > Add, Thunderbolt Bridge.
>
> I think the thunderbolt bridge is a mac-only thing.
>
> > When you plug in the Thunderbolt cable that may
> > also give you a Thunderbolt Ethernet (Slot 0 or 1 in my case).
>
> No additional NIC devices appear in Device Manager when connecting the
> two laptops via TB4 cable, no extra USB4 endpoints appear on the

It seems like there's various posts which say MS broke Thunderbolt
networking in Windows, although they differ in which W10 build they claim
broke it.

> > If it doesn't, Add again and see if it's in there.
> both laptops have two thunderbolt ports, normally one laptop is
> connected to a TB4 dock, and the other port unused
>
> the second laptop normally is just powered over one port and the other
> used for occasional USB devices (though it does work as thunderbolt if
> plugged to the same dock as above)
>
> have tried undocking laptop so the only ports in use are between the two
> laptops with a cable.
>
> Discovered there is a utility called "Thunderbolt Control Centre" in the
> Windows store, which has some sort of role in authenticating devices
> (presumably to prevent the remote-DMA type attacks from firewire days?)
> it shows both ports, but no devices that it can authenticate.

You will need that. Thunderbolt authentication is a thing where Windows
pops up a message saying 'you plugged in an X, do you want to allow it Y/N?'
You can set it in BIOS to prompt, just do Displayport only (no PCIe data),
or disable the security.

This is separate to the communication over the TB link: that doesn't get
started until you click Y at the prompt.

You could try disabling Thunderbolt security in BIOS to see if it makes a
difference. I would also check BIOS to see if there are any other settings
that disable TB features.

> I think I've got a spare thunderbolt downstream port on the dock, might
> try the second laptop into that, but seems a bit unlikely it'd work that
> way when direct port->port doesn't.

Agreed, but I'd try it anyway.

> Very little seems to be talked about thunderbolt networking, apart from
> physical ethernet dongles, by the laptop manufacturers or microsoft :-(

Even the manufacturers support don't have a clue:
https://forums.anandtech.com/threads/thunderbolt-networking-not-showing-up-in-windows-10.2610815/#post-40959136

Theo

[Andy Burns]

[This seemed to get lost in transit, watch it show up twice now]

Theo wrote:

> Andy Burns wrote:
>

>> I've seen videos of it being demoed mac->pc with talk of pc->pc, and I
>> have no doubt it works mac->mac
>
> It does exist pc->pc, or at least did.
> https://www.reddit.com/r/Thunderbolt/comments/yae2bv/ethernetoverthunderbolt_connection_between/

Yes, I saw that post.

I went into DeviceManager with the environment variable
DEVMGR_SHOW_NONPRESENT_DEVICES=1 set and view hidden devices enabled,
there were some "ghosted" Thunderbolt Networking devices, but they never
became active when the cable was connected

Tried everything nothing worked, got desperate so I ended-up completely
blowing away all mention of thunderbolt in DeviceManager, removing all
trace of the drivers from disk, finding the most recent Win11 update
that includes thunderbolt drivers from Intel, newer than those from Lenovo

<https://www.catalog.update.microsoft.com/ScopedViewInline.aspx?updateid=e3c6df59-c7de-4dec-ba93-56d7017c9bd3#Overview>

extracted and manually installed all the tbt*.inf files (the setup.bat
omits to install the tbtp2pndisdrv.inf driver) and then it works!!

Each end shows up as a 20Gbps ethernet NIC to the other, just using
APIPA addresses, windows file sharing on a 9.8GB zip file gets about
620MB/s (so about 5Gbps)

I tried bumping up jumbo frames to 64kB but that made it slower, will
see what it can do with iperf tomorrow ...

[Andy Burns]

Andy Burns wrote:

> Each end shows up as a 20Gbps ethernet NIC to the other, just using
> APIPA addresses, windows file sharing on a 9.8GB zip file gets about
> 620MB/s (so about 5Gbps)

source machine has PCIe gen4 x4 NVMe
dest machine has PCIe gen3 x4 NVMe
so with windows file sharing, the storage was the limiting factor (some
assistance from spare RAM as 20GB cache on source side and 8GB buffer on
dest side)

Using iperf3, single-threaded it gets about 7.5Gbps (rather peaky
throughput)
and two-threaded just over 8.5Gbps (pretty flat throughput)
more threads than two don't give any more performance, so the 20Gbps is
rather theoretical.

In round terms 1GB per second, shame it's only a 2ft cable.

That was all with direct cable laptop1<->laptop2
same performance going via downstream port on thunderbolt dock

+-->3k monitor
|
laptop1<->dock<->laptop2
|
+-->4k monitor

> I tried bumping up jumbo frames to 64kB but that made it slower, will
> see what it can do with iperf tomorrow ...

Apparently It doesn't like going above 4084 byte jumbo frames.

Presumably 12 byte MAC addr overhead per 4kB frame of
ethernet-in-thunderbolt encapsulation?