How does kqueue handle the following case?

[Patrick Nip]

This is the Q&A section of man epoll

-bash-4.1$ man epoll
...

Q6 Will closing a file descriptor cause it to be removed from
all epoll sets automatically?

A6 Yes, but be aware of the following point. A file
descriptor is a reference to an open file description (see open(2)).
Whenever a descriptor is duplicated via dup(2),
dup2(2), fcntl(2) F_DUPFD, or fork(2), a new file
descriptor referring to the same open file description is created. An
open file description continues to exist until
all file descriptors referring to it have been closed.
A file descriptor is removed from an epoll set only after all the file
descriptors referring to the underlying
open file description have been closed (or before if the
descriptor is explicitly removed using epoll_ctl() EPOLL_CTL_DEL).
This means that even after a file descriptor
that is part of an epoll set has been closed, events may
be reported for that file descriptor if other file descriptors
referring to the same underlying file description
remain open.


In particular my question is what would the cache "kqtree" would do in
the above instance? Should the user somehow need to explicit remove
the tree node themself before calling close()? And how can one do that
through using the kqueue interface?

[Patrick Nip]

Sorry I meant to copy this instead:


o If using an event cache...

If you use an event cache or store all the file descriptors
returned from epoll_wait(2), then make sure to provide a way to mark
its closure dynamically (i.e., caused by a
previous event’s processing). Suppose you receive 100 events
from epoll_wait(2), and in event #47 a condition causes event #13 to
be closed. If you remove the structure and
close(2) the file descriptor for event #13, then your event
cache might still say there are events waiting for that file
descriptor causing confusion.

One solution for this is to call, during the processing of
event 47, epoll_ctl(EPOLL_CTL_DEL) to delete file descriptor 13 and
close(2), then mark its associated data struc-
ture as removed and link it to a cleanup list. If you find
another event for file descriptor 13 in your batch processing, you
will discover the file descriptor had been pre-
viously removed and there will be no confusion.

When a socket is close while it is being monitored by kqueue what
would happen? Should the user somehow need to explicit remove the tree

node themself before calling close()? And how can one do that through
using the kqueue interface?

[Mark Heily]

I think the answer for libkqueue is the same as for epoll_wait(2). If you are
consuming multiple kevents in a single call to kevent(2), it is your
responsibility to discard events that reference a file descriptor after it has
been closed.

libkqueue does not internally implement an event cache. If you call kevent()
asking to consume 100 events, it will internally call epoll_wait() and ask for
100 events.

According to epoll(4) there is no need to call epoll_ctl(EPOLL_CTL_DEL) to
remove a descriptor after it has been closed:

Q6 Will closing a file descriptor cause it to be removed from all
epoll sets automatically?

A6 Yes...

However, you may be right to suggest that libkqueue needs to internally detect
the socket close event and delete the associated knote. I don't see anything
in the code to handle this situation. I will write a testcase to check if the
knote is deleted after the descriptor is closed.

There was a similar fix committed in r551 for the EVFILT_VNODE filter. The
same type of thing will probably need to be done for EVFILT_READ and EVFILT_WRITE.

Thanks,

- Mark

On 07/30/2012 04:52 PM, Patrick Nip wrote:
> Sorry I meant to copy this instead:
>
>
> o If using an event cache...
>
> If you use an event cache or store all the file descriptors
> returned from epoll_wait(2), then make sure to provide a way to mark
> its closure dynamically (i.e., caused by a

> previous event�s processing). Suppose you receive 100 events

[Patrick Nip]

Gave it a little more thought in the subject. In my opinion, may be
the best way to handle epoll managed resources should be a transparent
pass through your cache layer. Meaning your layer should only
responsible for mapping the kqueue interface to epoll interface but
once that is done and you have register your event in epoll, you
should never persist a knote. And on copyout, you should serialize a
kevent object dynamically based on the information you have gotten
back from epoll.

> > previous event s processing). Suppose you receive 100 events