Seeking users to try file monitoring package

[Sage Gerard]

Hi all,

I'm working on a filesystem monitoring package to contribute to the community.

https://github.com/zyrolasting/file-watchers

This is my first package and foray into threads with Racket, so there's much to improve. I'd appreciate your feedback to help me understand more about what the community looks for in this kind of work.

I'm particularly curious if the (robust-watch) procedure provided by the package in particular behaves consistently across operating systems. I do not have a Mac and was hoping that a Mac+Racket user would be willing to try this out and report his or her experience. This package is not yet published and has to be cloned.

Thank you in advance. Happy to address any questions.

-slg

[Simon Schlee]

I Just had a quick look at the documentation, it states:

watch-directories
Starts threads using thread-maker to watch each given directory. Meant for use with file-watcher procedures, namely apathetic-watch, intensive-watch, or robust-watch. This will block the current thread until all threads created with thread-maker terminate.

My understanding is that this watches for a single change for every directory and then unblocks. Creating a thread for each directory seems unnecessary (see example below).

Usually my filesystem-watching-needs are of a different nature:

- Get informed about changed files in a directory or one of its subdirectories, and what that changed file is

- do something with the filepath for that file (e.g. calculate the new sha1 hash compare with the old one, etc.)

- continue watching for other changes

To accomplish this I usually use a hash to map monitored pathes to the filesystem-change-evt that monitors that path.

If you use wrap-evt you can make your life easier by returning the path as the syncronization result instead of the filesystem-change-evt itself:

(define (wrap path)
  (wrap-evt (filesystem-change-evt path)
            (λ (evt) path)))

Then you can monitor all directories with one big sync, that returns one of the changed pathes, then you can immediately create a new filesystem-change-evt for that path to get further changes.

Here is an example:

#lang racket

(define (wrap path)
  (wrap-evt (filesystem-change-evt path)
            (λ (evt) path)))

(define (watch-pathes pathes proc)
  (define state
    (for/hash ([p (in-list pathes)])
      (values p (wrap p))))

  (define watcher
    (thread
     (thunk
      (let loop ()
        (define changed-path (apply sync (hash-values state)))
        (set! state (hash-update state changed-path (λ (old) (wrap changed-path))))
        ;; at this point the change events are all updated and ready to detect new changes
        ;; so it is not problematic if we do not sync right away
        ;; send changed-path to some queue/async-channel/small&quick lambda etc.
        (proc changed-path)
        (loop)))))

  (define (stop)
    (kill-thread watcher))

  stop)


(module+ main
  (require racket/async-channel)

  (define changed-pathes (make-async-channel))
  (define watcher
    (watch-pathes (list "." "..")
                  (λ (path)
                    (async-channel-put changed-pathes path))))

  (define tmp "./tmp.txt")
  (with-output-to-file tmp #:mode 'text (thunk (displayln "temp temp temp")))
  (sleep 1)
  (delete-file tmp)

  (let loop ()
    (define path (async-channel-get changed-pathes))
    (when path
      (displayln (~a "Directory has changed: " path))
      (loop))))

;; In a full example you would also handle when filesystem-change-evt fails with an error
;; if the path does not exist, but that seems to be higher-level logic
;; and quite dependend on the application

From the documentation I don't quite get what the typical usecase for your watch-directories function would be, because I would have to call the function multiple times if I want to get more than one change notification per directory.

And if I specify two directories and one has many changes but the second only 1 after an hour, the application would only react to the changes in the first directory after a long time.

I am a Linux user so I can't help you with the mac testing.

Well after looking into the code I realize that you already do something similar with bulk-filesystem-change-evt and apathetic-watch.

And the example for apathetic-watch seems quite clear, but I think you want kill-thread instead of thread-wait ?
Anyways I will leave the example because it is already written, maybe it is somewhat useful.

[Simon Schlee]

After some more reading I think I understand the purpose for watch-directories as a simple way to define callbacks for changes.

Maybe it could return a shutdown procedure but I guess using a custom custodian and shutting that down might be easier.

[Ben Greenman]

> I'm particularly curious if the (robust-watch) procedure provided by the
> package in particular behaves consistently across operating systems. I do
> not have a Mac and was hoping that a Mac+Racket user would be willing to try
> this out and report his or her experience.

(robust-watch) worked for me.

I'm on OSX El Capitan 10.11.1

Here's the terminal output:

$ racket main.rkt -m robusto ../a
Unrecognized method: #<procedure:parameter-procedure>. Falling back to
robust watch.
Starting robust watch over paths:
--> ../a

(robust add ../a/file.txt)
(robust add ../a/dirrr)
(robust change ../a/file.txt)
(robust remove ../a/file.txt)
(robust remove ../a)