Watching the file system for changes using vert.x?

[Stephan Wissel]

Hi there,
I want to watch certain files on my file system and send updates/changes using a websockets. The websockets parts is a breeze with vert.x, but I'm struggling conceptually with the WatchService integration. There is an example on the Oracle site.  Looking at that code, especially:

void processEvents() {
        for (;;) {

makes me wonder: that looks very much like very suitable for the vert.x event queue (?). Where would I put such code? In the main loop, into a worker? Is there an example of WatchService used in vert.x?

[Daryl Teo]

You could replace the for(;;) loop with a vertx periodic task.

I suggested on a couple of occasions integrating the WatchService stuff into FileSystem. Another item on the todo list I guess. =)

Regards,

Daryl

[Norman Maurer]

Never do this :)

Just use a a periodic task for it.

-- 
Norman Maurer

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Stephan Wissel]

I thought...

I call a verticle that wraps around the WatchService and spins of a worker. The worker then calls back with a change event to the main loop and at the end of the change event the verticle is called again, so I never call if nothing happens.

[Jordan Halterman]

I'd still go with a periodic task. Is there any reason you need change events to be handled with that type of accuracy instead of within a second delay or something?

Jordan Halterman

[Stephan Wissel]

Actually yes. Imagine 2 Windows next to each other (one an editor, non browser) one a browser. User hits save in the editor. The browser needs to refresh after some Java transformation 

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/5kOaEZxsWjk/unsubscribe.
To unsubscribe from this group and all of its topics, send an email to vertx+un...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Create a nice day!
Stephan H. Wissel

Phone: +65 96673269

Blog Twitter Google+ Facebook LinkedIn Xing Sametime Skype

[Daryl Teo]

Surely that can be triggered by the save itself, and not the updating of the file.

Daryl

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/5kOaEZxsWjk/unsubscribe.

To unsubscribe from this group and all of its topics, send an email to vertx+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[santo...@gmail.com]

Actually you should combine the periodic task and the for loop where the for loop is only used for looping over the list of newly received events since previous schedule, not for endless looping
(There is still some debugging stuff in here)

  @Override
    public void start() {

        try {           
// Set up the watcher
            watcher = FileSystems.getDefault().newWatchService();
            logger.trace(logPrefix + "register all watchdirs");
            for (Object dir : watchDirs) {
                Path dirPath = Paths.get(System.getProperty("user.dir") + File.separator + dir);
                WatchKey dirKey = dirPath.register(watcher, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_DELETE, StandardWatchEventKinds.ENTRY_MODIFY);
                if (logger.isDebugEnabled()) {
                    logger.debug(logPrefix + "Watcher registered for: " + dirPath.toString());
                }
                watchKeys.put(dirKey, dirPath);
            }

// periodically check for new events:
           vertx.setPeriodic(500, new Handler<Long>() {
                @Override
                public void handle(Long timerId) {
                    try {
                        processEvents();
                    }
                    catch (IOException exc) {
                        logger.error(logPrefix + "I/O error occured while processing events: " + exc.getLocalizedMessage() + "(cause: " + exc.getCause() + ")");
                    }
                    catch (Exception exc) {
                        logger.error(logPrefix + "Unexpected error occured while processing events: " + exc.getLocalizedMessage());
                    }
                }
            });
        }
        catch (ClosedWatchServiceException exc) {
            logger.warn(logPrefix + "The service was closed, exiting...");
        }
    }

    private void processEvents() throws IOException, Exception {
        for (;;) {
            WatchKey key;
            key = watcher.poll();           
             // watch key is null if no queued key is available (within the specified timeframe if a timeout was specified on the poll() request)
             if (key == null) break;

             logger.info(logPrefix + "Events received, start processing... (key: " + key + ")");

             Path dir = watchKeys.get(key);
             if (dir == null) {
                 logger.warn(logPrefix + "watchKey not recognized! (" + key.toString());
                 continue;
             }

             for (WatchEvent<?> event: key.pollEvents()) {
                 logger.info(logPrefix + "processing changes for: " + dir.toString());
                 WatchEvent<Path> watchEvent = (WatchEvent<Path>) event;
                 WatchEvent.Kind<Path> kind = watchEvent.kind();

                 if (kind.name().equals(StandardWatchEventKinds.OVERFLOW.name())) {
                     continue;
                 }

                 //The filename is the context of the event.
                 Path filename = watchEvent.context();
                 JsonObject watchMsg = new JsonObject();
                 // Add filename to message
                 watchMsg.putString("filename", filename.toString());
                 // Add file location (dir) to message
                 watchMsg.putString("location", dir.toString());
                 // Add type of event (create, modify, delete)
                 watchMsg.putString("type", kind);

                 // publish on eventbus
                 vertx.eventBus().publish("app.filechanges, watchMsg);
             }

             //Reset the key -- this step is critical if you want to receive
             //further watch events. If the key is no longer valid, the directory
             //is inaccessible so exit the loop.
             boolean valid = key.reset();
             if (!valid) {
                 logger.warn(logPrefix + "watchKey invalidated, removing from the list (" + key + ")");
                 watchKeys.remove(key);

                 // Exit if no keys remain
                 if (watchKeys.isEmpty()) {
                     break;
                 }
             }
        }
    }

[Jez P]

Only if the editor codebase is under Stephan's control. I suspect that he's talking about tracking changes which could be made with any editor, not one he owns the codebase for. 

[Tim Fox]

We used to use WatchService for our deployer functionality - we never got it to work and it was buggy in certain OS's (especially OSX, iirc) - so I'd avoid WatchService like the plague.

Have a look in Redeployer.java to see how it works now (without WatchService)

[Stephan Wissel]

exactly - I don't control the editor - it might be interactive or arrive via the network (CIFS, FTP, SCP) or a deamon process.

I'll have a look at Redeployer.java

Create a nice day!
Stephan H. Wissel

Phone: +65 96673269

Blog Twitter Google+ Facebook LinkedIn Xing Sametime Skype

To unsubscribe from this group and all of its topics, send an email to vertx+un...@googlegroups.com.