Re: [Caml-list] Looking for stubs for sendmsg/recvmsg
Jérémie Dimino
> I'm looking for stubs for
>
> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>
> Specifically I need those to send (among normal messages) an
> Unix.file_descr over a Unix Domain Socket.
>
> Does anyone know of a module that has them?
The Lwt_unix module has them [1].
Cheers,
Jérémie
[1] http://ocsigen.org/lwt/doc/api/Lwt_unix.html#VALrecv_msg
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Sylvain Le Gall
> Hi,
>
> I'm looking for stubs for
>
> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>
> Specifically I need those to send (among normal messages) an
> Unix.file_descr over a Unix Domain Socket.
>
> Does anyone know of a module that has them?
>
If you don't find one and plan to write it yourself, this would be a
good addition to extunix:
http://extunix.forge.ocamlcore.org
Regards,
Sylvain Le Gall
Dave Scott
>
> I'm looking for stubs for
>
> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>
> Specifically I need those to send (among normal messages) an
> Unix.file_descr over a Unix Domain Socket.
>
> Does anyone know of a module that has them?
We have some bindings for those:
https://github.com/xen-org/xen-api-libs/blob/master/stdext/unixext.mli
external send_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag list -> Unix.file_descr -> int = "stub_unix_send_fd_bytecode" "stub_unix_send_fd"
external recv_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag list -> int * Unix.sockaddr * Unix.file_descr = "stub_unix_recv_fd"
You might prefer to extract the relevant functions from the code -- there's a lot of other misc stuff in that repo which you're probably not interested in.
We use those functions quite a lot so hopefully they'll work for you.
Cheers,
Dave
Goswin von Brederlow
> On 18-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>> Hi,
>>
>> I'm looking for stubs for
>>
>> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
>> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>>
>> Specifically I need those to send (among normal messages) an
>> Unix.file_descr over a Unix Domain Socket.
>>
>> Does anyone know of a module that has them?
>>
>
> If you don't find one and plan to write it yourself, this would be a
> good addition to extunix:
> http://extunix.forge.ocamlcore.org
>
> Regards,
> Sylvain Le Gall
I'm thinking of changing Unix.file_descr from int to a custom block
(containing the FD) with finalizer. Unix.close would set the FD to -1
and the finalizer gives an error if FD != -1 and closes it.
Actually I want that tunable with 3 possible behaviours:
type fd_leak_mode = Silent | Complain | Fail
val set_leak_mode : fd_leak_mode -> unit = <fun>
Silent just closes the FD if it is still open, Complain (default)
outputs to stderr and closes it and Fail aborts.
That would change most of the Unix module and mean a complete fork of it.
Would that be something for extunix too?
MfG
Goswin
Goswin von Brederlow
>> Hi,
>>
>> I'm looking for stubs for
>>
>> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
>> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>>
>> Specifically I need those to send (among normal messages) an
>> Unix.file_descr over a Unix Domain Socket.
>>
>> Does anyone know of a module that has them?
>
> We have some bindings for those:
>
> https://github.com/xen-org/xen-api-libs/blob/master/stdext/unixext.mli
>
> external send_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag list -> Unix.file_descr -> int = "stub_unix_send_fd_bytecode" "stub_unix_send_fd"
> external recv_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag list -> int * Unix.sockaddr * Unix.file_descr = "stub_unix_recv_fd"
>
> You might prefer to extract the relevant functions from the code -- there's a lot of other misc stuff in that repo which you're probably not interested in.
>
> We use those functions quite a lot so hopefully they'll work for you.
>
> Cheers,
> Dave
Neigther one is directly suitable but cut&paste into my own stub is easy
enough with that. Thanks Dave and Jérémie.
MfG
Goswin
Sylvain Le Gall
On 19-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
> Sylvain Le Gall <syl...@le-gall.net> writes:
>
>> On 18-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>> Hi,
>>>
>>> I'm looking for stubs for
>>>
>>> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
>>> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>>>
>>> Specifically I need those to send (among normal messages) an
>>> Unix.file_descr over a Unix Domain Socket.
>>>
>>> Does anyone know of a module that has them?
>>>
>>
>> If you don't find one and plan to write it yourself, this would be a
>> good addition to extunix:
>> http://extunix.forge.ocamlcore.org
>>
>> Regards,
>> Sylvain Le Gall
>
> I'm thinking of changing Unix.file_descr from int to a custom block
> (containing the FD) with finalizer. Unix.close would set the FD to -1
> and the finalizer gives an error if FD != -1 and closes it.
>
> Actually I want that tunable with 3 possible behaviours:
>
> type fd_leak_mode = Silent | Complain | Fail
> val set_leak_mode : fd_leak_mode -> unit = <fun>
>
> Silent just closes the FD if it is still open, Complain (default)
> outputs to stderr and closes it and Fail aborts.
>
> That would change most of the Unix module and mean a complete fork of it.
Not that much, if you proceed in another way. I think what you are
looking for is a fd leak detector?
Here is a small modules that I have written for this purpose:
File UnixExt.ml:
(** Count open/close call *)
IFNDEF NDBUG THEN
let fd_opened =
Hashtbl.create 13
;;
let fd_once_opened =
Hashtbl.create 13
;;
let fd_open fd fn out =
dbug_print
(fun () ->
Printf.sprintf "%s '%s'"
(if out then "open-out" else "open-in")
fn);
Hashtbl.add fd_opened fd (fn, out)
;;
let fd_close fd =
try
let (fn, out) as data =
Hashtbl.find fd_opened fd
in
dbug_print
(fun () ->
Printf.sprintf "%s '%s'"
(if out then "close-out" else "close-in")
fn);
Hashtbl.add fd_once_opened fd data;
Hashtbl.remove fd_opened fd;
with Not_found ->
begin
dbug_print
(fun () ->
let fn =
try
fst (Hashtbl.find fd_once_opened fd)
with Not_found ->
"unknown"
in
Printf.sprintf "Trying to close %s again" fn)
end
;;
let () =
at_exit
(fun () ->
let exit_error =
ref false
in
Hashtbl.iter
(fun fd (fn, out) ->
if fd <> Unix.stdin && fd <> Unix.stdout && fd <> Unix.stderr then
begin
Printf.eprintf "Not closed '%s' (out: %b)\n" fn out;
exit_error := true
end)
fd_opened;
Hashtbl.clear fd_opened;
if !exit_error then
exit 3
)
;;
let opened_files () =
let lst =
ref []
in
Hashtbl.iter
(fun _ e -> lst := e :: !lst)
fd_opened;
List.sort compare !lst;
ELSE
let fd_open _ _ _ =
()
;;
let fd_close _ =
()
;;
let opened_files () =
[]
;;
ENDIF
(** See UnixExt.mli *)
let to_file_descr_in fd =
fd_open fd "<converted>" false;
fd
;;
(** See UnixExt.mli *)
let to_file_descr_out fd =
fd_open fd "<converted>" true;
fd
;;
(** See UnixExt.mli *)
let close_in fd =
Unix.close fd;
fd_close fd
;;
(** See UnixExt.mli *)
let stdout =
fd_open Unix.stdout "<stdout>" true;
Unix.stdout
;;
[...override other functions that open/close fd...]
Then in the modules using this features, you just have to open UnixExt
after Unix...
You can even probably design a library that will transparently hide Unix
with a custom Unix module providing this feature.
>
> Would that be something for extunix too?
>
I don't think so. At least, this is not currently the purpose of
extunix...
Regards,
Sylvain Le Gall
Goswin von Brederlow
Much less usefull.
Using a custom block with finalizer means that the FD will be closed
relative close to where/when it was leaked. Makes it easier to find
where it was leaked and adding GC.compact calls at strategic locations
can narrow it down even more. Leaking FDs also becomes much less
serious. The GC will clean them up and close them. So you can use an app
that leaks FDs just fine.
MfG
Goswin
Sylvain Le Gall
> Sylvain Le Gall <syl...@le-gall.net> writes:
>> On 19-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>> Sylvain Le Gall <syl...@le-gall.net> writes:
>>>> On 18-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>
>> Not that much, if you proceed in another way. I think what you are
>> looking for is a fd leak detector?
>>
>> Here is a small modules that I have written for this purpose:
>>
>> File UnixExt.ml:
>> (** Count open/close call *)
[...]
>> [...override other functions that open/close fd...]
>>
>> Then in the modules using this features, you just have to open UnixExt
>> after Unix...
>>
>> You can even probably design a library that will transparently hide Unix
>> with a custom Unix module providing this feature.
>
> Much less usefull.
>
> Using a custom block with finalizer means that the FD will be closed
> relative close to where/when it was leaked. Makes it easier to find
> where it was leaked and adding GC.compact calls at strategic locations
> can narrow it down even more. Leaking FDs also becomes much less
> serious. The GC will clean them up and close them. So you can use an app
> that leaks FDs just fine.
>
It all depends on what you want: fix your program that leaks or live
with it. The former piece of code helps to fail if there are leaked FD.
On Unix FD leaks is not that problematic, but on Windows it turns to be
another problem.
The best example about this: you cannot delete a file that has an FD
still open on it. This makes harder to remove temporary file (and this
piece of code was precisely made to track FD on temporary files, that
let 1000s of unremoved temp file).
Regards,
Sylvain Le Gall
Goswin von Brederlow
Which again speaks for my solution. The leaked FD will be closed much
faster (before the program terminates) and one can remove the tempfiles
while the program is still running.
MfG
Goswin
Sylvain Le Gall
> Sylvain Le Gall <syl...@le-gall.net> writes:
>> On 19-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>> Sylvain Le Gall <syl...@le-gall.net> writes:
>>>> On 19-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>>>> Sylvain Le Gall <syl...@le-gall.net> writes:
>>>>>> On 18-11-2010, Goswin von Brederlow <goswi...@web.de> wrote:
>>
>> The best example about this: you cannot delete a file that has an FD
>> still open on it. This makes harder to remove temporary file (and this
>> piece of code was precisely made to track FD on temporary files, that
>> let 1000s of unremoved temp file).
>>
>> Regards,
>> Sylvain Le Gall
>
> Which again speaks for my solution. The leaked FD will be closed much
> faster (before the program terminates) and one can remove the tempfiles
> while the program is still running.
>
I think, this totally off-topic. But anyway, when a program create a
temporary files it needs to remove ASAP, itself. I wouldn't
deliver a program with a notice like "sometimes FD leaked, not a
problem, just remove $TMP/myprogram-*".
For the little story, the leaked FD (hence the temporary files) was
400MB each and it quickly get noticed after a few run (and a full HD).
Regards,
Sylvain Le Gall
Goswin von Brederlow
Consider the following code:
(* Make sure everything is finalised at exit *)
let () = at_exit Gc.full_major
let remove_tempfile name _ =
Unix.unlink name
let fd = Unix.openfile name ...
in
Gc.finalise (remove_tempfile name) fd
(* do whatever with the FD *)
This ensures the tempfile is closed and removed once it becomes
unreachable. One doesn't need to catch all exceptions to ensure the file
is removed even on errors. Or need reference counting if the FD is
referenced multiple times.
Currently the above fails because Unix.file_descr (int) isn't heap
allocated.
I'm not saying it is a solves-all-problems solution. I just found it
verry helpfull with other abstract types that needed to be explicitly
closed before they can be discarded.
MfG
Goswin
ygrek
Sylvain Le Gall <syl...@le-gall.net> wrote:
> >
> > Would that be something for extunix too?
> >
>
> I don't think so. At least, this is not currently the purpose of
> extunix...
From implementation point of view this means that ExtUnix is a functor over
file_descr type and C side uses specific macro instead of Int_val - not a big deal.
But in order to be useful all functions from the Unix module should be duplicated -
not very funny.
--
ygrek
http://ygrek.org.ua