Strange behavior reading UDP packets

[Boris]

This might has nothing to do with Go.

I've created a simple UDP listener. I'm not sure what it does, but what I want is read data and print them on stdout. Here's the code:

func main() {                                                          

   service := ":8032"                                                  

   conn, err := net.ListenPacket("udp", service)                       

   if err != nil { fmt.Println(err) }                                                   

   for {                                                               

      var buf [512]byte                                              

      n, _ /*addr*/, err := conn.ReadFrom(buf[0:])                     

      fmt.Println("Read bytes:", n)                                    

      if err != nil {                                                  

         fmt.Println(err)                                              

      } else {                                                         

         fmt.Println(string(buf[0:n]))                                 

      }                                                                

   }                                                                   

}  

When I use another program to send it a lot of data, say 7000 bytes, it reads and prints out 512 bytes as expected (and the other program reports sending 7000 bytes). But first it prints a lot of newlines. This is too many bytes to show nicely in this email, so I'll use a shorter analogy. Imagine:

1. The sending program sends the letters of the alphabet, each on a line.

2. The listening program has a buffer that's big enough for a-h.

3. The listening program prints out a lot of newlines, followed by a-h.

I'm really puzzled by this. Can anyone explain it to me?

Beyond my puzzlement, can someone comment if I'm daft in my general approach? My thought was that if someone sends my listener a huge chunk of data, I'll be able to read it 512 bytes at a time off the connection and reassemble the big chunk. I am not a network expert, so I have some questions I don't know how to answer, such as will there be problems if lots of senders are sending data concurrently, will I miss some of the data if things get too busy, and so forth.

Thanks,

Boris.

[Jan Mercl]

On Fri, Nov 2, 2012 at 6:13 PM, Boris <boris.s...@gmail.com> wrote:

Few quick thougths:

func main() {                                                          

   service := ":8032"                                                  

   conn, err := net.ListenPacket("udp", service)                       

   if err != nil { fmt.Println(err) }                                                   

It's nice to print the error, but the overall logic is broken here. You cannot reasonably continue using the `conn` bellow if `err` is not nil. You're effectively ignoring any errors here.

 

   for {                                                               

      var buf [512]byte                                              

      n, _ /*addr*/, err := conn.ReadFrom(buf[0:])                     

      fmt.Println("Read bytes:", n)                                    

      if err != nil {                                                  

         fmt.Println(err)                                              

      } else {                                                         

         fmt.Println(string(buf[0:n]))                                 

This is IMO the probable source of "it prints a lot of newlines" - when `n` is zero. It _may_ be an artifact of the previously ignored error - in theory.

 

      }                                                                

   }                                                                   

}  

And: From the top of my head I think that UDP has no transport ability, so if your buffer is smaller than the transmitted packet, you can never get the "overrun" part anymore - but maybe the "net" package (or kernel?) takes care of that for you (really not sure) - I never tried to "continue" in such situation.

-j

[Boris Solovyov]

Hi Jan,

> func main() {
>>
>> service := ":8032"
>> conn, err := net.ListenPacket("udp", service)
>> if err != nil { fmt.Println(err) }
>
>
> It's nice to print the error, but the overall logic is broken here. You
> cannot reasonably continue using the `conn` bellow if `err` is not nil.
> You're effectively ignoring any errors here.

Agreed, I should change that in a non-throwaway program.

>> for {
>> var buf [512]byte
>> n, _ /*addr*/, err := conn.ReadFrom(buf[0:])
>> fmt.Println("Read bytes:", n)
>> if err != nil {
>> fmt.Println(err)
>> } else {
>> fmt.Println(string(buf[0:n]))
>
>
> This is IMO the probable source of "it prints a lot of newlines" - when `n`
> is zero. It _may_ be an artifact of the previously ignored error - in
> theory.

I think you are saying that ReadFrom reads 0 bytes in a loop, and
occasionally reads some data when I send it from the other program.
This isn't what is happening, because I modified my program to print a
begin/end banner around the printed data from the buffer. If it were
doing what you suggest, then I would have a lot of banners with an
empty line between each one. ReadFrom() is blocking until it reads
some bytes.

As for the error - there isn't any error printed, so I don't think there is one.

I should have taken the advice at
http://www.catb.org/esr/faqs/smart-questions.html and asked a smarter
question with a better code sample :-) Thanks for your patience and
help.

Thanks
Boris

[Mikio Hara]

On Sat, Nov 3, 2012 at 2:13 AM, Boris <boris.s...@gmail.com> wrote:

> I'm really puzzled by this. Can anyone explain it to me?

I guess, because your sender program writes so.
Pasted toy sender, listener below work fine.

http://play.golang.org/p/A_sLlueHMr
http://play.golang.org/p/6PmdIYRAVn

> Beyond my puzzlement, can someone comment if I'm daft in my general
> approach? My thought was that if someone sends my listener a huge chunk of
> data, I'll be able to read it 512 bytes at a time off the connection and
> reassemble the big chunk.

ReadFrom on PacketConn, UDPConn uses syscall recvfrom.
So please have a look at the on-line manual.

man 2 recvfrom on Unix variants says: If a message is too long to fit
in the supplied buffer, excess bytes may be discarded depending on
the type of socket the message is received from.

Hope this helps,
-- Mikio

[Boris Solovyov]

On Sun, Nov 4, 2012 at 9:07 PM, Mikio Hara <mikioh...@gmail.com> wrote:
> Pasted toy sender, listener below work fine.
>
> http://play.golang.org/p/A_sLlueHMr
> http://play.golang.org/p/6PmdIYRAVn

Thank you Mikio. Your examples are very helpful. Not only am I doing
it "wrong" but I don't know how to write idiomatic Go yet.

- Boris