[CommonJS] Sockets API proposal

[Mário Valente]

I tought I'd make myself useful and contribute to CommonJS in ways other
than lurking and voting...

... and I also have a personal interest in having both the Filesystem API
and the Socket AP available
and implemented ASAP :-)

So:

- I created a survey of Socket APIs available...

http://spreadsheets.google.com/pub?key=tYP9b4a7U5ezmh1ivHURJXg

- Evaluating the above, plus shameless copying, pasting and pure personal
opinion, I created
a proposal (A) for the sockets API

http://wiki.commonjs.org/wiki/Sockets/A

- I modified the wiki homepage (moved Sockets from Future Effors to
Current Efforts) and
reformatted the Sockets page including adding the link to Proposal A

Its been a long time since I've done plain sockets programming so lets
hope I dont make a
fool of myself. There is definitely some stuff missing:

- aditional functionalities, either methods or properties
- functions/names/properties from other engines addressing aditional
functionality (developers
please provide documentation links)
- corrections and aditions taking into account the Java sockets API
(Java experts please feel
free to mock my total Java ignorance)

Feel free to add directly to the wiki pages, but I will try to keep
abreast and sum up corrections and
suggestions and add them to the wiki

-- MV

[Ash Berlin]

Sockets are certainly something we'll want to spec.

A few comnents:

> Returns success of the operation (TRUE|FALSE)

We have exceptions in javascript - we should be throwing on failures, not requiring the programmer to check errors explicitly

> receiveFrom( [maxlen])

Missing host/port in the func sig?

> sendTo

This should probably be on the Socket constructor, not a socket instance, since its a UDP packet and has all the info needed in the call itself.

> sendFile(file)

I don't see the need for this, mainly cos I can't see it being all that useful in practice. It kind of dilutes things.

> size ... Determine the amount of number of bytes waiting to be read on the socket - READ ONLY

I'm not sure that is always available.

> type ...  SOCK_STREAM (TCP) or SOCK_DGRAM (UDP)

There are other types of sockets available on various platforms :)

More generally I think (connected TCP and UDP) sockets should use the same stream interface that comes out of the various IO proposals. They can have extra metadata, but having open sockets and open file handles have the same read/write interfaces gives a nice symmetry to the CommonJS environment to my mind.

-ash

[Wes Garland]

A few quick comments:

 - Sockets should work like Streams -- I am not even sure if they should be differentiable except by feature-testing
 - close() and shutdown() should be separate
 - shutdown() should support SHUT_RD, SHUT_WR and SHUT_RDWR functionality
 - Why no support for AF_UNIX?
 - It may be desirable to add network-interface option to bind() for multi-homed machines with multiple IP addresses per NIC
 - Timeouts should be specified in microseconds (tm.tv_usec)
 - I do not see a way to do OOB
 - send/receive are superflous if socket is a Stream, would be read/write
 - I do not understand the point of receiveFrom. Is it for raw sockets?
 - Same with sendTo. Does it open a socket, dump data, and close the socket?
 - IME/IIUC sendFile is used in webservers to implement no-copy data transmission where network interface and disk hardware share a common buffer. This is a highly implementation-specific optimization and probably doesn't belong in a general sockets API.
 - set option / get option: IMO, numbers should not be allowed. Are they guaranteed constant across platforms?
 - Tests: see Stream() comments
 - size: this is not knowable in the general case
 - Meta data should include protocol  (e.g. TCP)

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

[Daniel Friesen]

Note while I'm iPod limited.

I'll be doing some Societ specing as part of IO/B.

~Daniel Friesen (DanTMan, Nadir Seen Fire) [http://daniel.friesen.name]

[Daniel Friesen]

Btw since you're Filesystem interested I'm working on a level 0
filesystem in IO/B/Filesystem/Level0 which can likely be standardized
much quicker than a full system (and leaves room to experiment with
ideas in a way that works on all platforms whether standardized or not).

~Daniel Friesen (DanTMan, Nadir Seen Fire) [http://daniel.friesen.name]

[Mário Valente]

On Sun, Sep 20, 2009 at 5:46 PM, Ash Berlin <ash_flu...@firemirror.com> wrote:

A few comnents:

> Returns success of the operation (TRUE|FALSE)

We have exceptions in javascript - we should be throwing on failures, not requiring the programmer to check errors explicitly

  OK, corrected on all instances.



 

> receiveFrom( [maxlen])

Missing host/port in the func sig?

  Not really... This method is for unconnected sockets and thus returns an object with
 data read plus host and port. Connected sockets should use receive() and in that
 case the host/port data is on the socket instance.




 

> sendTo

This should probably be on the Socket constructor, not a socket instance, since its a UDP packet and has all the info needed in the call itself.

   Yes, in the case of a connected socket. In that case you'd use send(). In the case of an unconnected socket
 you dont have the host/port info at the socket object and he binding is done temporarily when sendTo is called.



 

> sendFile(file)

I don't see the need for this, mainly cos I can't see it being all that useful in practice. It kind of dilutes things.

   Its present in most APIs. And I guess that the alternative for sending a huge amount of info (stored in a
 file) would be to read the file block by block and sending.



 

> size ... Determine the amount of number of bytes waiting to be read on the socket - READ ONLY

I'm not sure that is always available.

  OK, removed.



 

> type ...  SOCK_STREAM (TCP) or SOCK_DGRAM (UDP)

There are other types of sockets available on various platforms :)

  Yes, but I challenge you to give me actual usage of RAW, RDM and SEQPACKET :-)
  These can, of course, be included.



 

More generally I think (connected TCP and UDP) sockets should use the same stream interface that comes out of the various IO proposals. They can have extra metadata, but having open sockets and open file handles have the same read/write interfaces gives a nice symmetry to the CommonJS environment to my mind.

    Yes, I agree. But given that IO proposals and stream interfaces are still being defined, I think
 that work could/should go forward in sockets. And then its just the case of changing some
 method names (receive->read, send->write, etc). I will gladly receive possible changes to the
 socket interface so as to become Stream compatible and will incorporate those changes. For
 now I'll keep them as it is.


    -- MV

[Mário Valente]

On Sun, Sep 20, 2009 at 9:05 PM, Wes Garland <w...@page.ca> wrote:

A few quick comments:

 - Sockets should work like Streams -- I am not even sure if they should be differentiable except by feature-testing

   Agreed. But right now we dont have the Streams interface defined. I'll keep the Socket interface as is for now.
   Later the interface can be made identical with the Streams interface. Meanwhile I'll take any suggestions on what
  should be changed (send->write?, receive->read?)


 

 - close() and shutdown() should be separate

   Changed.

 

 - shutdown() should support SHUT_RD, SHUT_WR and SHUT_RDWR functionality

   Changed.


 

 - Why no support for AF_UNIX?

   Changed.

 

 - It may be desirable to add network-interface option to bind() for multi-homed machines with multiple IP addresses per NIC

   Its there already. "host" is the IP address of the chosen interface to which the socket should bind.


 

 - Timeouts should be specified in microseconds (tm.tv_usec)

  Changed.


 

 - I do not see a way to do OOB

  
    Yes. Like SOCKET_RAW, SEQPACKET or RDM I guess I'm guessing that it might be difficult
 to find examples of OOB being used :-). Its also poorly documented and there are compatibility
 issues between different protocol stack implementations.
    Do you think its really needed?

 

 - send/receive are superflous if socket is a Stream, would be read/write

   Indeed. But (see above) I guess that it would depend on the Stream interface. I've put
 them in as alternatives for now.

 

 - I do not understand the point of receiveFrom. Is it for raw sockets?

   Unconnected UDP sockets. Returns data and info about remote host and port


 

 - Same with sendTo. Does it open a socket, dump data, and close the socket?

   Sends data to a specific host/port on an unconnected UDP socket.



 

 - IME/IIUC sendFile is used in webservers to implement no-copy data transmission where network interface and disk hardware share a common buffer. This is a highly implementation-specific optimization and probably doesn't belong in a general sockets API.

   Its present in most general sockets APIs. Having it, even if its not the zero-copy version, certainly makes
 things better than doing a block-by-block read and write or read data into an array/BLOB and sending that.
  

 

 - set option / get option: IMO, numbers should not be allowed. Are they guaranteed constant across platforms?

    Changed.

 

 - Tests: see Stream() comments

   Agreed. See also above. Changeable once we get a stable Stream interface. Will gladly
 receive suggestions.



 

 - size: this is not knowable in the general case

   Indeed, I've already removed it

 

 - Meta data should include protocol  (e.g. TCP)

    Its there already: type (last line of the page)


    -- MV
 

[Irakli Gozalishvili]

Hi,

I was wondering if anybody is still interested in ratifying Socket  API's. I don't think I can propose something new here but I would like to push to get this topic rolling as I do use Socket's in my commonjs related project and would be really happy to be compilant with the specs.

Regards
--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

2009/10/7 Mário Valente <mfva...@gmail.com>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en
-~----------~----~----~----~------~----~------~--~---

[Daniel Friesen]

If you want, I have an old Socket draft for MonkeyScript
http://draft.monkeyscript.org/api/io/Socket.html

Comes from the same mindset as IO/B
http://wiki.commonjs.org/wiki/IO/B

It'll likely be what I base IO/B/Socket off of.
If you want you can look at IO/B, that old draft, and your own api to
see what it's missing.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Irakli Gozalishvili wrote:
> Hi,
>
> I was wondering if anybody is still interested in ratifying Socket
> API's. I don't think I can propose something new here but I would
> like to push to get this topic rolling as I do use Socket's in my
> commonjs related project and would be really happy to be compilant
> with the specs.
>
> Regards
> --
> Irakli Gozalishvili
> Web: http://rfobic.wordpress.com/
> Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands
>
>

> 2009/10/7 M�rio Valente <mfva...@gmail.com <mailto:mfva...@gmail.com>>

> <mailto:comm...@googlegroups.com>

> To unsubscribe from this group, send email to
> commonjs+u...@googlegroups.com

> <mailto:commonjs%2Bunsu...@googlegroups.com>

> For more options, visit this group at
> http://groups.google.com/group/commonjs?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
>

> --

>
> You received this message because you are subscribed to the Google
> Groups "CommonJS" group.

> To post to this group, send email to comm...@googlegroups.com.

> For more options, visit this group at

> http://groups.google.com/group/commonjs?hl=.

[Ash Berlin]

On 19 Nov 2009, at 01:22, Irakli Gozalishvili wrote:

> Hi,
>
> I was wondering if anybody is still interested in ratifying Socket
> API's. I don't think I can propose something new here but I would
> like to push to get this topic rolling as I do use Socket's in my
> commonjs related project and would be really happy to be compilant
> with the specs.
>
> Regards

I'd like sockets. A socket should look like a stream. So until we have
a ratified stream proposal we can't really do sockets. Also I had my
reservations about this sockets proposal: can't remember what now, but
I felt some unease about this proposal when i last looked. (Which
admittedly was months ago now)

-ash

[Wes Garland]

> I'd like sockets. A socket should look like a stream. So until we have
> a ratified stream proposal we can't really do sockets.

Agreed. 1000%.

On the subject of streams, my recent work shows that making streams generators is impractical (albeit cool on paper). It's too hard to do incremental processing in subroutines where the increments are bigger than a single yield.

Wes