I was very tangentially involved with a use-after-close IO descriptor fault recently, and I came to realize that descriptor indexes are typically allocated by the smallest available index
Block the current thread until data is available to read on the given file descriptor (GHC only).
This will throw an IOError
if the file descriptor was closed while this thread was blocked. To safely close a file descriptor that has been used with threadWaitRead
, use closeFdWith
.
So what should happen in your situation is that, when a separate thread closes the fd, threadWaitRead will throw an exception promptly, not continue to wait on a now-closed fd.
Note the docs for closeFdWith:
Close a file descriptor in a concurrency-safe way (GHC only). If you are using threadWaitRead
or threadWaitWrite
to perform blocking I/O, you must use this function to close file descriptors, or blocked threads may not be woken.
So I think the short answer is, if you're using fd's and threadWaitRead, and you always use closeFdWith, you should be protected against operating on a re-used fd index, even with wrapping your fd's in a Maybe.
I seem to recall that there are some other issues that arise if you attempt to mix these low-level fd calls with higher-level fd operations, but that was two IO managers ago so those concerns may be out of date.
John L.
_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
I seem to recall that there are some other issues that arise if you attempt to mix these low-level fd calls with higher-level fd operations, but that was two IO managers ago so those concerns may be out of date.
There are *always* issues with mixing low-level and high-level I/O, especially if the high-level I/O is buffered. It's best to not mix them. (Note that using handleToFd will safely give you something that you can do low level I/O with and will close the Handle; fdToHandle can't stop you from mixing them, though.)On Mon, Sep 1, 2014 at 7:13 PM, John Lato <jwl...@gmail.com> wrote:
I seem to recall that there are some other issues that arise if you attempt to mix these low-level fd calls with higher-level fd operations, but that was two IO managers ago so those concerns may be out of date.
So what should happen in your situation is that, when a separate thread closes the fd, threadWaitRead will throw an exception promptly, not continue to wait on a now-closed fd.
I was thinking you could do away entirely with the Maybe wrapper. But I guess that might not be possible in your case.
Have you actually observed this behavior? I suspect that the thread will never yield between readMVar and threadWaitRead because there are no allocations. I suppose an async exception could arise, so it might be correct to run that line with exceptions masked.
John
I was thinking you could do away entirely with the Maybe wrapper. But I guess that might not be possible in your case.
Have you actually observed this behavior? I suspect that the thread will never yield between readMVar and threadWaitRead because there are no allocations. I suppose an async exception could arise, so it might be correct to run that line with exceptions masked.
I don't see how one could allow concurrent readers and "closers" withoutleaving this small opening. The best workaround I can think of is to
create a blocking close operation that waits for readers using a semaphore.
On Tue, Sep 2, 2014 at 1:31 PM, Felipe Lessa <felipe...@gmail.com> wrote:
I don't see how one could allow concurrent readers and "closers" withoutleaving this small opening. The best workaround I can think of is to
create a blocking close operation that waits for readers using a semaphore.
Well yes, I can't think of a simple lock-based solution to this problem, because you don't want to hold any kind of lock while you are blocked on the file descriptor.
It would be solvable though if we had a thread-safethreadWaitReadMVar :: MVar Fd -> IO ()You should be able to implement this relatively easily if you dig deeper into the IO manager itself, but we already have threadWaitReadSTM. (For different reasons, which doesn't cover this use case.) How many variations on this theme are we going to need.We could implement threadWaitReadMVar if we had a non-blocking way of registering interest in a file descriptor, and then later actually blocking on it. So let's say something like
registerWaitRead :: Fd -> IO (IO ())threadWaitReadMVar fd = join $ withMVar fd registerWaitReadWhich, ignoring asynchronous exceptions for the moment, should be adequate for the task. I suppose that means you could instead dothreadWaitReadMVar fd = (atomically . fst) =<< withMVar fd threadWaitReadSTM
Which seems like an odd use of STM, but that also does seem like a solution. So I guess the earlier part of this email (as well as eariler emails) is in fact wrong, that threadWaitReadSTM does cover this use case. And STM might also offer a nicer way of making multiple reads from the inotify binding thread-safe as well.
On Tue, Sep 2, 2014 at 12:26 PM, Leon Smith <leon.p...@gmail.com> wrote:
On Tue, Sep 2, 2014 at 1:31 PM, Felipe Lessa <felipe...@gmail.com> wrote:
I don't see how one could allow concurrent readers and "closers" withoutleaving this small opening. The best workaround I can think of is to
create a blocking close operation that waits for readers using a semaphore.Well yes, I can't think of a simple lock-based solution to this problem, because you don't want to hold any kind of lock while you are blocked on the file descriptor.If you wanted to go this route, you could use an MVar (Maybe (Int,Fd)), where the Int is a count of interested threads. Instead of using readMVar before threadWaitRead, you would use modifyMVar to atomically increment the counter and retrieve the fd. Then, after threadWaitRead returns, decrement the counter. You'd need to make sure that you never close an fd when the counter is greater than 0. This would work better with a TMVar, because then the close operation could block until the counter has changed.
It would be solvable though if we had a thread-safethreadWaitReadMVar :: MVar Fd -> IO ()You should be able to implement this relatively easily if you dig deeper into the IO manager itself, but we already have threadWaitReadSTM. (For different reasons, which doesn't cover this use case.) How many variations on this theme are we going to need.We could implement threadWaitReadMVar if we had a non-blocking way of registering interest in a file descriptor, and then later actually blocking on it. So let's say something like
registerWaitRead :: Fd -> IO (IO ())threadWaitReadMVar fd = join $ withMVar fd registerWaitReadWhich, ignoring asynchronous exceptions for the moment, should be adequate for the task. I suppose that means you could instead dothreadWaitReadMVar fd = (atomically . fst) =<< withMVar fd threadWaitReadSTMWhich seems like an odd use of STM, but that also does seem like a solution. So I guess the earlier part of this email (as well as eariler emails) is in fact wrong, that threadWaitReadSTM does cover this use case. And STM might also offer a nicer way of making multiple reads from the inotify binding thread-safe as well.This should work even in the presence of async exceptions, barring bugs in withMVar and threadWaitReadSTM. You can implement your registerWaitRead using forkIO and MVars, but I think the only reason to do so would be for compatibility with older ghcs. It's probably more sensible to just copy the definition of threadWaitReadSTM in that case, unless you want to target pre-STM compiler versions.John
On Tue, Sep 2, 2014 at 12:26 PM, Leon Smith <leon.p...@gmail.com> wrote:
On Tue, Sep 2, 2014 at 1:31 PM, Felipe Lessa <felipe...@gmail.com> wrote:
I don't see how one could allow concurrent readers and "closers" withoutleaving this small opening. The best workaround I can think of is to
create a blocking close operation that waits for readers using a semaphore.Well yes, I can't think of a simple lock-based solution to this problem, because you don't want to hold any kind of lock while you are blocked on the file descriptor.If you wanted to go this route, you could use an MVar (Maybe (Int,Fd)), where the Int is a count of interested threads. Instead of using readMVar before threadWaitRead, you would use modifyMVar to atomically increment the counter and retrieve the fd. Then, after threadWaitRead returns, decrement the counter. You'd need to make sure that you never close an fd when the counter is greater than 0. This would work better with a TMVar, because then the close operation could block until the counter has changed.
threadWaitReadMVar fd = (atomically . fst) =<< withMVar fd threadWaitReadSTM
This should work even in the presence of async exceptions, barring bugs in withMVar and threadWaitReadSTM. You can implement your registerWaitRead using forkIO and MVars, but I think the only reason to do so would be for compatibility with older ghcs. It's probably more sensible to just copy the definition of threadWaitReadSTM in that case, unless you want to target pre-STM compiler versions.