Synchronization Issue

[rionm...@gmail.com]

I've been playing around with using a single container that contains multiple viewports that I add a "synced" attribute to in order to iterate through them and add them to a synchronizer. However, it doesn't seem to be working as expected.

I'm only using the scrollStack tool for example purposes and the behavior that I am looking for is "when one of the synced elements is scrolled, all of the elements will scroll up", however the current behavior seems to have all of the elements scrolling independently (e.g. if you scroll the first viewport to the 50th image, when you try to scroll another one of the elements, it will begin at the 50th element). So it seems that the actual index is being synced between the viewports, but the actual viewports do not all scroll at the same time (which is the expected behavior).

A rough example of the code that I am using to wire up this syncronizer looks like the following :

// Create a syncronizer

imageViewer.syncronizers.push(new cornerstoneTools.Synchronizer("CornerstoneNewImage", cornerstoneTools.stackImageIndexSynchronizer));

// Activate all of the view ports (within the new template that was added)

$('.viewport.synced', viewportTemplate).each(function () {

       var element = $(this).get(0);

       cornerstone.enable(element);

       // Populate the number of view ports that need to be populated

       if (imageIndex < $('.viewport.synced', viewportTemplate).length && imageIndex < activeStack.imageIds.length) {
              // Store the current instance on the element

              $(this).attr('data-instance', imageIndex);

              // Pull the specific image to be loaded for this stack (relative to the appropriate position)

              var image = imageViewer.images[activeStack.imageIds[imageIndex++]];

              // Add this viewport to the collection of viewports (along with the stack / series it is associated with)

              imageViewer.viewports.push({ Viewport: element, Series: activeStack });

              // Display the image on the viewer element

              if (image != undefined) {

                    // Load the image and wire up the appropriate tools, adding the element to the syncronizer

                    cornerstone.displayImage(element, image);

                    cornerstoneTools.addStackStateManager(element, ['stack']);

                    cornerstoneTools.addToolState(element, 'stack', activeStack);

                    cornerstoneTools.stackScroll.activate(element, 1);

                    cornerstoneTools.stackScrollWheel.activate(element);

                    imageViewer.syncronizers[0].add(element);

              }

              // Details omitted for brevity

       }  

});

When comparing this to the available example within the tooling, it seems that all of the similar behaviors are being taken : 

// Define these elements (equivalent to the synced viewports above)

var axialElement1 = $('#axial1').get(0);

var axialElement2 = $('#axial2').get(0);

var axialElement3 = $('#axial3').get(0);

// Grab the image ids from the stack (present in the activeStack above)

var axialImageIds = [ 'example://1', 'example://2']

var axialStack1 = { currentImageIdIndex : 0, imageIds: axialImageIds };

var axialStack2 = { currentImageIdIndex : 0, imageIds: axialImageIds };

var axialStack3 = { currentImageIdIndex : 0, imageIds: axialImageIds };

// Define a syncronizer to use (present in the imageViewer.syncronizers collection above)

var synchronizer = new cornerstoneTools.Synchronizer("CornerstoneNewImage", cornerstoneTools.stackImageIndexSynchronizer);

// Enable each element

cornerstone.enable(axialElement1);

cornerstone.enable(axialElement2);

cornerstone.enable(axialElement3);

// Enable elements for this element

cornerstoneTools.mouseInput.enable(axialElement1);

cornerstoneTools.mouseWheelInput.enable(axialElement1);

cornerstoneTools.mouseInput.enable(axialElement2);

cornerstoneTools.mouseWheelInput.enable(axialElement2);

cornerstoneTools.mouseInput.enable(axialElement3);

cornerstoneTools.mouseWheelInput.enable(axialElement3);

// Images are already loaded within the imageViewer.images collection (so grab them)

cornerstone.loadImage(axialImageIds[0]).then(function(image) {

        // display this image (equivalent to being called within the loop) 

        cornerstone.displayImage(axialElement1, image);

        cornerstone.displayImage(axialElement2, image);

        cornerstone.displayImage(axialElement3, image);

        // set the stack as tool state (being set above as well)

        cornerstoneTools.addStackStateManager(axialElement1, ['stack']);

        cornerstoneTools.addStackStateManager(axialElement2, ['stack']);

        cornerstoneTools.addStackStateManager(axialElement3, ['stack']);
        // Is the issue that these are all targeting the same stack? Should they be independent elements?

        cornerstoneTools.addToolState(axialElement1, 'stack', axialStack1);

        cornerstoneTools.addToolState(axialElement2, 'stack', axialStack2);

        cornerstoneTools.addToolState(axialElement3, 'stack', axialStack3);

        // Enable all tools we want to use with this element

        cornerstoneTools.stackScroll.activate(axialElement1, 1);

        cornerstoneTools.stackScrollWheel.activate(axialElement1);

        cornerstoneTools.stackScroll.activate(axialElement2, 1);

        cornerstoneTools.stackScrollWheel.activate(axialElement2);

        cornerstoneTools.stackScroll.activate(axialElement3, 1);

        cornerstoneTools.stackScrollWheel.activate(axialElement3);

        // Done above as well

        synchronizer.add(axialElement1);

        synchronizer.add(axialElement2);

        synchronizer.add(axialElement3);
}

Is there anything in particular that looks to be missing or that I should be looking into as to why each of the elements may not be all syncing as expected. Any ideas or insight would be helpful and I would be glad to try and provide some additional examples if needed.

Thanks,

Rion

[Chris Hafey]

Hi Rion,

I didn't exhaustively review your code, but it looks like you are using the synchronizer correctly.  I don't quite understand what the exact bug is though - maybe you can try explaining it again with more detail?  Note that the synchronizers have not been well tested or used yet so there could be some bugs lurking.

Chris

[rionm...@gmail.com]

Alright, I'll give it another shot with a bit more detail.

What I am currently doing is adding the functionality for right clicking a single viewport and breaking it into multiple, smaller synced viewers (e.g. 2x2, 3x3, etc.). For now, I've just thrown together a very basic double-click/tap event that will trigger this and it's working as expected as when I do this, the following occurs :

1. Double-click or double-tap on an existing viewport (specifically one that is not already defined as "synced" through a data attribute).
   
   
   $('.viewport').on('dblclick', function () {
           // Code omitted (as it will be seen below)
   
   
   
2. Metadata for the current viewport is read (e.g. current series, etc.) and then the tools for the viewport are disabled and the viewport is finally disabled and removed from the DOM.
   
   
   var _viewportClicked = $(this).get(0);
   // Get the previous viewport number
   var activeSeries = $(this).data('series');
   var activeStack = imageViewer.stacks[parseInt($(this).data('series'))];
   var previousViewport = $(_viewportClicked).attr('data-viewport');
   // Disable all of the tools for this current viewport
   disableAllToolsForViewport(_viewportClicked);
   // Update the layout here
   cleanupEnabledElements();
   // The layout has changed, clear out the view port collection
   
   imageViewer.viewports = imageViewer.viewports.filter(function (vp) { return vp.Viewport != _viewportClicked; });
   
   
3. A template (e.g. 2x2) is loaded into the location of the previous viewport, a "synced" attribute is added to each of the viewports in the template along with series information.
   
   
   // Read from the selected template
   var viewportTemplate = $($('#synced-viewport-2x2').clone().html());
   $(this).replaceWith(viewportTemplate);
   
   
   // Indicate on the wrapper the current series
   $(viewportTemplate).attr('data-series', activeSeries);
   $(viewportTemplate).attr('data-previous-viewport', previousViewport);
   
   
4. A syncronizer is created at this point.

1. 
   
   
   // Create a syncronizer
   imageViewer.syncronizers.push(new cornerstoneTools.Synchronizer("CornerstoneNewImage", cornerstoneTools.stackImageIndexSynchronizer));
   
   

2. Each of the newly created synced viewports are iterated through, enabled, have their specific tooling enabled (just scroll stack for example purposes), image displayed and finally and are added to the syncronizer
   
   
   // Activate all of the view ports (new viewports)

1. $('.viewport.synced', viewportTemplate).each(function () {
               var element = $(this).get(0);
               cornerstone.enable(element);
               // Populate the number of view ports that need to be populated
               if (imageIndex < $('.viewport.synced', viewportTemplate).length && imageIndex < activeStack.imageIds.length) {

1.                 $(this).attr('data-instance', imageIndex);
                   // Pull the current image

1.                 var image = imageViewer.images[activeStack.imageIds[imageIndex++]];
   
   
                   // Add this viewport to the collection of viewports (along with the stack / series it is associated with)
                   imageViewer.viewports.push({ Viewport: element, Series: activeStack });
   
   
                   // Display the image on the viewer element
                   if (image != undefined) {

1.                     cornerstone.displayImage(element, image);
                       cornerstoneTools.addStackStateManager(element, ['stack']);
                       cornerstoneTools.addToolState(element, 'stack', activeStack);
                       cornerstoneTools.stackScroll.activate(element, 1);
                       cornerstoneTools.stackScrollWheel.activate(element);
                       imageViewer.syncronizers[0].add(element);
                   }
               }

1. });

I tried to follow the same basic code within the CornerstoneTools example that was used here, which is working as expected (e.g. any stack scrolling on one of the elements will scroll the stack on each of them). However, the behavior that I am experiencing is as follows :

1. Begin scrolling on one of the viewports, it scrolls independently. Let's say for example's sake you scroll to the 25th image.
2. If you hover over another one of the other available synced viewports and begin scrolling, it will jump to the 25th image from it's current position and continue on (as if you were still on the first viewport that was targeted).

So it appears as though the actual position is being syncronized properly (e.g. they are all on the same index within the series), but only the viewport that is being actively scrolled over is being updated. Hopefully that provides a bit more clarification.

[rionm...@gmail.com]

Just as a follow-up,

Although I have this functionality in my current implementation, it's likely going to vary depending on what users want to do with it (i.e. I use things like HTML data attributes to manage my current positions in the series on a viewport-basic along with some other data), however I've prepared a very basic more generic implementation in a fork but I need some input.

The synchronizer is designed to do exactly what it sounds like. It synchronizes any scrolling operations between a series of viewports and propagates the changes across all of them. For instance, if you have three synced viewports and they hold images 1, 2, 3 respectively, and then you scroll upwards on the first element, 1 will become 2, 2 becomes 3 and so forth. One of the ways that this was achieved was by extended the existing synchronizer.js to support the passing along of eventData through the following two changes (highlighted below) :

function fireEvent(sourceEnabledElement, eventData) {

            // Broadcast an event that something changed

            ignoreFiredEvents = true;

            $.each(targetElements, function (index, targetEnabledElement) {

                handler(that, sourceEnabledElement, targetEnabledElement, eventData);

            });

            ignoreFiredEvents = false;

}

function onEvent(e, eventData) {

            if (ignoreFiredEvents === true) {

                return;

            }

            fireEvent(e.currentTarget, eventData);

}

Now in order to extend any synchronizer, you simply need to add an eventData parameter to it's function and you'll be able to access data that was passed along inside : 

function stackScrollSynchronizer(synchronizer, sourceElement, targetElement, eventData) {

        // If there is no event, ignore synchronization

        if (eventData == undefined) {

            return;

        }

}

You can see an example of it in action here, however there is one glaring issue, which I'm asking for input on. One of the changes that was made to this was adding a trigger for each possible scrolling event (stackScroll, stackScrollKeyboard, etc.) that looks like the following : 

function mouseWheelCallback(e, eventData) {

        var images = -eventData.direction;

        cornerstoneTools.scroll(eventData.element, images);

        $(eventData.element).trigger("CornerstoneImageScroll", {direction :images});

}

The problem here is that the built-in scroll event (which eventually calls scrollToIndex) contains logic which might cause an "overflow" to occur and reset the currentIndex of the stack to 0. This behavior can throw off the synchronization and cause things to get severely out of wack. 

A few possible ideas or resolutions might be :

• Explicitly ignore the scroll event for elements that are contained within a synchronizer.
• Define a different tool for handling scrolls on specific synchronized elements.
• Store the previous index either within the stack or on the element to support "overflow detection"

I know in my own implementation, I've overridden the scrollToIndex function with a custom one that checks if the viewport has a CSS class called "synced" and it allows the synchronizer to take care of the scrolling on it's own : 

cornerstoneTools["scrollToIndex"] = customScrollToIndex;

function customScrollToIndex(element, newImageIdIndex) {

    // Omitted for brevity

    if (newImageIdIndex !== stackData.currentImageIdIndex) {

        stackData.currentImageIdIndex = newImageIdIndex;

        var viewport = cornerstone.getViewport(element);

        // It's synced, let the synchronizer handle it

        if ($(element).hasClass('synced')) {

            return;

        }

        // Otherwise load the image

        cornerstone.loadAndCacheImage(stackData.imageIds[newImageIdIndex]).then(function (image) { ... });   

    }

}

Any input is more than welcome. Additionally, if someone could look at the example, very bizarre things appear to be going on with the actual images themselves (window levels are erratically changing with scrolling, etc.).