UDP Broadcast

[Mr Horn]

I am successfully using unicast with the net package's ListenUDP(),
WriteTo() and ReadFrom() API [1]. However, when I switch to broadcast
addresses such as 255.255.255.255 and the correct port number, the
receiver does not pick up the messages anymore.

Has anyone run into similar issues?

To investigate the problem, I have looked at fd.go [2] where the flags
for the Recvfrom syscall [3] are zero. Is it possible that the
receiving of broadcast messages should be explicitly enabled as part
of the system call?

Cheers,
Alex

[1] http://golang.org/pkg/net/#UDPConn
[2] http://golang.org/src/pkg/net/fd.go#L371
[3] http://golang.org/pkg/syscall/#Recvfrom

[Dave Cheney]

Hi Alex,

I've written a few programs that use UDP broadcast, this is how I use it

-- client

BROADCAST_IPv4 := net.IPv4(255, 255, 255, 255)
socket, err := net.DialUDP("udp4", nil, &net.UDPAddr{
IP: BROADCAST_IPv4,
Port: port,
})

-- server

socket, err := net.ListenUDP("udp4", &net.UDPAddr{
IP: net.IPv4(0, 0, 0, 0),
Port: port,
})
for {
data := make([]byte, 4096)
read, remoteAddr, err := socket.ReadFromUDP(data)
}

Cheers

Dave

[Alex Horn]

Based on Dave's generous feedback, I have been able to verify that I
can now successfully broadcast a message provided that I open the
socket with DialUDP() instead of ListenUDP(). Initially, I was under
the impression that I could also simply open the socket with
ListenUDP() and use WriteTo() with a broadcast address. Instead, the
suggestion involving DialUDP() and Write() seems to be the only way to
broadcast a message. Is this the intentional library behavior?

A potential use case where such behavior could seem counterintuitive
is in a P2P application over UDP, for example. In such a setting, all
peers may wish to initially just listen for incoming packets using
ListenUDP(). The sending of a message is only triggered by (say) user
intervention on a particular host. Call this host A. Then, in this use
case, host A would want to use WriteTo() in order to broadcast
something. The benefit of this approach includes the possibility of
(re)using the socket for other transmissions such as a unicast to a
particular peer in the network if necessary.

Alex

[James Nurmi]

A little late to the game, but I have successfully used listenudp, but
I'm explicitly using readfromudp, rather than readfrom (sorry, my
api-docs are away from me). I'm also doing this as root (low port #),
so I have 'extra' permissions when testing.

Notes to remember w/r/t broadcast: Your os MUST permit traffic to be
routed to the address (ip route get 255.255.255.255), if it doesn't,
you'll need a more specific broadcast address (network broadcast might
be something like 10.255.255.255 or 192.168.123.255, etc).

I'm binding to a specific interface (udpconn.BindToInterface) which
(IIRC my C) makes broadcasts easier to accomplish on linux.

You still have to bind to a local address (0.0.0.0 is fine, but the
responder must broadcast as well if your client has no IP.)

If you need more ptrs, I'll try and work up some test cases, but
hopefully something here might jog a successful broadcast for you.

-James

[Johann Höchtl]

On Jan 13, 7:57 am, James Nurmi <jdnu...@qwe.cc> wrote:
> A little late to the game, but I have successfully used listenudp, but
> I'm explicitly using readfromudp, rather than readfrom (sorry, my
> api-docs are away from me).  I'm also doing this as root (low port #),
> so I have 'extra' permissions when testing.
>
> Notes to remember w/r/t broadcast: Your os MUST permit traffic to be
> routed to the address (ip route get 255.255.255.255), if it doesn't,
> you'll need a more specific broadcast address (network broadcast might
> be something like 10.255.255.255 or 192.168.123.255, etc).
>

BTW another gotcha: If you test your scanario over WIFI you're likely
to fail. UDP over WIFI is very implementation dependant and most of
the time does not work or with unacceptable latencies.

> I'm binding to a specific interface (udpconn.BindToInterface) which
> (IIRC my C) makes broadcasts easier to accomplish on linux.
>
> You still have to bind to a local address (0.0.0.0 is fine, but the
> responder must broadcast as well if your client has no IP.)
>
> If you need more ptrs, I'll try and work up some test cases, but
> hopefully something here might jog a successful broadcast for you.
>
> -James
>

> On Wed, Jan 12, 2011 at 1:31 PM, Alex Horn <alex.h...@gmail.com> wrote:
> > Based on Dave's generous feedback, I have been able to verify that I
> > can now successfully broadcast a message provided that I open the
> > socket with DialUDP() instead of ListenUDP(). Initially, I was under
> > the impression that I could also simply open the socket with
> > ListenUDP() and use WriteTo() with a broadcast address. Instead, the
> > suggestion involving DialUDP() and Write() seems to be the only way to
> > broadcast a message. Is this the intentional library behavior?
>
> > A potential use case where such behavior could seem counterintuitive
> > is in a P2P application over UDP, for example. In such a setting, all
> > peers may wish to initially just listen for incoming packets using
> > ListenUDP(). The sending of a message is only triggered by (say) user
> > intervention on a particular host. Call this host A. Then, in this use
> > case, host A would want to use WriteTo() in order to broadcast
> > something. The benefit of this approach includes the possibility of
> > (re)using the socket for other transmissions such as a unicast to a
> > particular peer in the network if necessary.
>
> > Alex
>

> > On 12 January 2011 20:28, Dave Cheney <d...@cheney.net> wrote:
> >> Hi Alex,
>
> >> I've written a few programs that use UDP broadcast, this is how I use it
>
> >> -- client
>
> >> BROADCAST_IPv4 := net.IPv4(255, 255, 255, 255)
> >> socket, err := net.DialUDP("udp4", nil, &net.UDPAddr{
> >>        IP:   BROADCAST_IPv4,
> >>        Port: port,
> >> })
>
> >> -- server
>
> >> socket, err := net.ListenUDP("udp4", &net.UDPAddr{
> >>        IP:   net.IPv4(0, 0, 0, 0),
> >>        Port: port,
> >> })
> >> for {
> >>        data := make([]byte, 4096)
> >>        read, remoteAddr, err := socket.ReadFromUDP(data)
> >> }
>
> >> Cheers
>
> >> Dave
>

[roger peppe]

On 13 January 2011 10:30, Johann Höchtl <johann....@gmail.com> wrote:
> BTW another gotcha: If you test your scanario over WIFI you're likely
> to fail. UDP over WIFI is very implementation dependant and most of
> the time does not work or with unacceptable latencies.

that's an interesting statement. have you got a reference for it?

[Johann Höchtl]

On Jan 13, 11:38 am, roger peppe <rogpe...@gmail.com> wrote:

> On 13 January 2011 10:30, Johann Höchtl <johann.hoec...@gmail.com> wrote:
>
> > BTW another gotcha: If you test your scanario over WIFI you're likely
> > to fail. UDP over WIFI is very implementation dependant and most of
> > the time does not work or with unacceptable latencies.
>
> that's an interesting statement. have you got a reference for it?

Only personal observation with differents devices implementing bonjour
which where behind different AP's in Client Mode.It has never been an
issue with the WIFI router but with devices in Client Mode. For UDP in
broadcast it would be necessary to constantly listen, which those
devices do not do.

Here's my serverfault question (without answers)
http://serverfault.com/questions/175393/udp-mdns-over-wifi-ap-in-client-mode

[Mr Horn]

Let's keep the discussion focused if possible about broadcasts on a
single LAN.

To help us pin down the problem, I have created a functional test [1].
Notice that if line seventeen [2] is changed to

addr, err := net.ResolveUDPAddr(":8200")

such that the requester is sending directly to the responder (without
involving any broadcasts), then the test passes.
However, if the requester does use the broadcast address
net.IPv4bcast, then the test hangs because the responder never sees
the incoming broadcast.
This failure occurs even if the user who is running the gotest is
root.

Alex

[1] https://github.com/ahorn/gossip/blob/7a818fc84a4303e79615ff6411a6ca75f7bdd7b4/broadcast_test.go
[2] https://github.com/ahorn/gossip/blob/7a818fc84a4303e79615ff6411a6ca75f7bdd7b4/broadcast_test.go#L17

> > On 12 January 2011 20:28, Dave Cheney <d...@cheney.net> wrote:
> >> Hi Alex,
>
> >> I've written a few programs that use UDP broadcast, this is how I use it
>
> >> -- client
>
> >> BROADCAST_IPv4 := net.IPv4(255, 255, 255, 255)
> >> socket, err := net.DialUDP("udp4", nil, &net.UDPAddr{
> >>        IP:   BROADCAST_IPv4,
> >>        Port: port,
> >> })
>
> >> -- server
>
> >> socket, err := net.ListenUDP("udp4", &net.UDPAddr{
> >>        IP:   net.IPv4(0, 0, 0, 0),
> >>        Port: port,
> >> })
> >> for {
> >>        data := make([]byte, 4096)
> >>        read, remoteAddr, err := socket.ReadFromUDP(data)
> >> }
>
> >> Cheers
>
> >> Dave
>

[Mr Horn]

Sorry the line number I meant to refer to can be found here [1].

[1] https://github.com/ahorn/gossip/blob/634a84570cfdf0ca6a03cb38ac67d2eb34b2bcc7/broadcast_test.go#L17

On Jan 13, 4:24 pm, Mr Horn <alex.h...@gmail.com> wrote:
> Let's keep the discussion focused if possible about broadcasts on a
> single LAN.
>
> To help us pin down the problem, I have created a functional test [1].
> Notice that if line seventeen [2] is changed to
>
>   addr, err := net.ResolveUDPAddr(":8200")
>
> such that the requester is sending directly to the responder (without
> involving any broadcasts), then the test passes.
> However, if the requester does use the broadcast address
> net.IPv4bcast, then the test hangs because the responder never sees
> the incoming broadcast.
> This failure occurs even if the user who is running the gotest is
> root.
>
> Alex
>

> [1]https://github.com/ahorn/gossip/blob/7a818fc84a4303e79615ff6411a6ca75...
> [2]https://github.com/ahorn/gossip/blob/7a818fc84a4303e79615ff6411a6ca75...

[roger peppe]

On 13 January 2011 11:30, Johann Höchtl <johann....@gmail.com> wrote:
>
>
> On Jan 13, 11:38 am, roger peppe <rogpe...@gmail.com> wrote:
>> On 13 January 2011 10:30, Johann Höchtl <johann.hoec...@gmail.com> wrote:
>>
>> > BTW another gotcha: If you test your scanario over WIFI you're likely
>> > to fail. UDP over WIFI is very implementation dependant and most of
>> > the time does not work or with unacceptable latencies.
>>
>> that's an interesting statement. have you got a reference for it?
>
> Only personal observation with differents devices implementing bonjour
> which where behind different AP's in Client Mode.It has never been an
> issue with the WIFI router but with devices in Client Mode. For UDP in
> broadcast it would be necessary to constantly listen, which those
> devices do not do.

oh, i'm sorry, your statement seemed to refer to UDP in general,
not particularly UDP broadcast.

[James Nurmi]

As sent, it doesnt build (net.ResolveUDPAddr)

Still, why are you resolving broadcast?

--- ../foo_test.go 2011-01-14 12:47:12.000000000 -0800
+++ t_test.go 2011-01-14 12:46:14.000000000 -0800
@@ -14,13 +14,9 @@
c := make(chan bool)
go respond(t, responder)
go receive(t, requester, c)
- addr, err := ResolveUDPAddr(net.IPv4bcast.String(), 8200)
- if err != nil {
- t.Fatalf("Cannot resolve broadcast address: %s", err)
- }

request := []byte("Some request")
- _, err = requester.WriteTo(request, addr)
+ _, err := requester.WriteTo(request, &net.UDPAddr{IP:net.IPv4bcast,
Port:8200})
if err != nil {
t.Fatalf("Unable to respond to request: %s", err)
}
@@ -71,3 +67,4 @@

return conn
}
+

passes on my system.

[Mr Horn]

> As sent, it doesnt build (net.ResolveUDPAddr)

My apologies; I included an incorrect link in the original post. Thank
you for having corrected it and running the tests. The outcome is
curious (see below).

> [t_test.go] passes on my system.

OK, these tests fail on my system. To prune the problem space, I was
able to verify that the broadcast in this test works correctly if I
change the "requester" socket to use DialUDP() [2]. What could be the
reason for this abnormality?

Cheers,
Alex

[2] https://github.com/ahorn/gossip/commit/17f5ed33086d9955ab05ae5c522130e373e63684#diff-0

On Jan 14, 8:49 pm, James Nurmi <jdnu...@qwe.cc> wrote:
> As sent, it doesnt build (net.ResolveUDPAddr)
>
> Still, why are you resolving broadcast?
>
> --- ../foo_test.go      2011-01-14 12:47:12.000000000 -0800
> +++ t_test.go   2011-01-14 12:46:14.000000000 -0800
> @@ -14,13 +14,9 @@
>         c := make(chan bool)
>         go respond(t, responder)
>         go receive(t, requester, c)
> -       addr, err := ResolveUDPAddr(net.IPv4bcast.String(), 8200)
> -       if err != nil {
> -               t.Fatalf("Cannot resolve broadcast address: %s", err)
> -       }
>
>         request := []byte("Some request")
> -       _, err = requester.WriteTo(request, addr)
> +       _, err := requester.WriteTo(request, &net.UDPAddr{IP:net.IPv4bcast,
> Port:8200})
>         if err != nil {
>                 t.Fatalf("Unable to respond to request: %s", err)
>         }
> @@ -71,3 +67,4 @@
>
>         return conn
>  }
> +
>
> passes on my system.
>
>
>
>
>
>
>

> On Thu, Jan 13, 2011 at 9:02 AM, roger peppe <rogpe...@gmail.com> wrote:

> > On 13 January 2011 11:30, Johann Höchtl <johann.hoec...@gmail.com> wrote:
>
> >> On Jan 13, 11:38 am, roger peppe <rogpe...@gmail.com> wrote:
> >>> On 13 January 2011 10:30, Johann Höchtl <johann.hoec...@gmail.com> wrote:
>
> >>> > BTW another gotcha: If you test your scanario over WIFI you're likely
> >>> > to fail. UDP over WIFI is very implementation dependant and most of
> >>> > the time does not work or with unacceptable latencies.
>
> >>> that's an interesting statement. have you got a reference for it?
>
> >> Only personal observation with differents devices implementing bonjour
> >> which where behind different AP's in Client Mode.It has never been an
> >> issue with the WIFI router but with devices in Client Mode. For UDP in
> >> broadcast it would be necessary to constantly listen, which those
> >> devices do not do.
>
> > oh, i'm sorry, your statement seemed to refer to UDP in general,
> > not particularly UDP broadcast.
>
>
>

>  t_test.go
> 2KViewDownload

[James Nurmi]

My first thoughts are back to the resolving of broadcast (might cause
some weirdness) and the interface routing.

- Double check that 'ip route get 255.255.255.255' works, and does
what you think it should.
- tcpdump both the routed interface (above) and the interface you
think it should be on, see which one actually gets the traffic
- tcpdump from another machine on the same physical subnet if possible
- check iptables & firewalling

I've gone ahead and published my GoDHCP[1] library ahead of time, as
this does a decent amount of work to get broadcast on (unbound)
interfaces correct without raw sockets, and might help you. While
it has no tests, and produces no stand-alone (normally), there is a
test_agent.go (you'll need to change the IFNAME and possibly the
HWADDR used to make it useful) that should at least work if you're on
any interface that doesn't have an existing dhclient. (It doesn't do
anything to your local host, simply does the DHCP negotiation, but an
existing client will probably catch at least some of the negotiation
replies).

Give it a try if you can, it's fairly well error-catching, and I'd
like to see what it spits out if nothing else in your environment.

[1] https://github.com/abneptis/GoDHCP

[Mr Horn]

With the Wireshark packet analyzer and tcpdump, I found out the
following:

1) The test with ListenUDP() [1] does broadcast to other hosts on the
LAN (e.g. 19:50:21.547353 IP mrh.xprint-server >
192.168.1.255.trivnet1: UDP, length 12) but the local ReadFrom()
goroutine never sees these packets. This phenomenon would make sense
if the local interface doesn't broadcast back to itself but I reckon
that someone else would have pointed out such an unsound assumption
earlier.

2) In contrast, the test which involves DialUDP() [2] passes on
localhost despite the fact there is no actual packets broadcast to
other hosts on the LAN (verifiable with tcpdump, for example).

I find the outcome of either experiment surprising. In fact, common
sense seems to suggest that at least one of these outcomes could point
to an underlying flaw. Finally, as a last resort, I also used James'
DHCP agent as inspiration and tried out BindToDevice(), for example.
No success.

Unless someone with more experience in network programming thinks what
has been described in this post could be a bug in the net library, I
understand if other more pressing issues take priority and we draw a
line here. I certainly thank everyone who has contributed to the
discussion. My hope is that the resolution of the issue is possible at
a later point in time.

Kind regards,
Alex

[1] https://github.com/ahorn/gossip/blob/791c2927c867c74c81e092309792fab7d47f7268/broadcast_test.go
[2] https://github.com/ahorn/gossip/blob/17f5ed33086d9955ab05ae5c522130e373e63684/broadcast_2_test.go

> > [2]https://github.com/ahorn/gossip/commit/17f5ed33086d9955ab05ae5c522130...

[jo...@hupla.com]

I'm very late to respond to this but it still seems to be a very minimal discussed topic.

I am trying to bind to a specific Interface (i have 4 on my server) and cannot use 0.0.0.0.

But when i try to use any other form of addr such as the enclosed 192.168.1.255 (class C) it runs but i never get messages from client broadcast 255.255.255.255 ??

Anyone have example like James mentioned on bindToAddress ?? 

func main() {

   port := 8001
   fmt.Println("Listening for UDP Broadcast Packet")

   socket, err := net.ListenUDP("udp4", &net.UDPAddr{
      IP:   net.IPv4(0, 0, 0, 0),

      //IP:   net.IPv4( 192, 168, 1, 255 ),
      Port: port,

   })

   if err != nil {

      fmt.Println("Error listen: " , err)

   }
   for {
      data := make([]byte, 4096)
      read, remoteAddr, err := socket.ReadFromUDP(data)

      if err != nil {
         fmt.Println("readfromudp: ", err)
      }
      for i :=0; i<read; i++{
         fmt.Println(data[i])
      }
      fmt.Printf("Read from: %v\n", remoteAddr)
   }

}