Questions about Cornerstone, Tools, Layouts and More

[rionm...@gmail.com]

Hi Chris,

Firstly, I was to let out a resounding thanks to you for this project (and the Cornerstone family of projects) as they have been extremely enlightening and helpful. I have previously worked on a project developing a non-Javascript based DICOM Viewer and my team and I have been waiting for a solid open-source project that had a similar focus to base a client-side version on.

In working with Cornerstone, I had the following questions that you might be able to assist me with (in some sense) :

At present, what would be the latest set of tools to use or a project to pull them from?

I know that I have seen some discrepancies with regards to support between CornerstoneDemo and some of the CornerstoneTools and was curious which to use.

One of the main issues I found was that some functionality (e.g. measurement tools) in the examples appeared to work in Internet Explorer, while the same tools did not in the Demo. I tried using some of the same example code found within the examples within the context of a Viewer, but still couldn't get them to work.

Do you have a recommendation for handling touch events? Are there some tools that shouldn't be expected to work with touch?

I've tried using various versions of hammer.js in hopes that it would pick up on some of the events, but I thought I would just check to see if you had any tips with regards to implementing it.

I know that some of the touch events such as window levels, zooming and panning all see to work as expected, but should I be able to expect something like angle measurements to work with a touch interface?

How are things progressing with regards to Layouts?

I know that this is obviously a very tough area to handle within a DICOM Viewer and handling things like reference lines and all that jazz must be a nightmare. I was just curious as to how things were progressing in that direction or if you had any type of working repository demonstrating it (or even a basic idea for it).

I believe those were the only questions that had come up thus far, but I figured that you would be far and away the best person to ask. Congratulations again on the SIIM 2015 Award as well, it's a tremendous accomplishment that you should certainly be proud of.

Thanks again and I look forward to hearing from you.

Cheers,

Rion

Rion Williams
Lead Software Developer / Microsoft MVP (ASP.NET/IIS)

[Chris Hafey]

Thank you for your kind words and welcome to the cornerstone community!  Here are some answers for you:

1) All tools development is done in the cornerstoneTools repository.  cornerstoneDemo references a specific version of cornerstoneTools via bower, no corernstoneDemo specific code is in that version. If you want the latest and greatest, get it from conerstoneTools.  Be aware that the tools library is under active development and bugs likely exist.

2) There have been some issues related to both IE and touch events but I don't know the current state of things - perhaps Erik Ziegler or Scott Poulin can chime in.  Search this forum for relevant threads

3) Touch has only been implemented for a few tools - nobody is working on adding touch to all of them AFAIK.  Contributions are welcome :)

4) I believe that one of cornerstone's key strengths when compared to other open source image viewers is that it restricts itself to the viewport level.  Layouts are actually pretty easy to do with HTML - simply create a DIV for each viewport and use CSS to size them properly.  Cornerstone does have support for multi viewport functionality such as reference lines and stack synchronization and there examples of both in the cornerstoneTools library.

Chris

[rionm...@gmail.com]

Thanks for the quick response Chris.

1) All tools development is done in the cornerstoneTools repository.  cornerstoneDemo references a specific version of cornerstoneTools via bower, no corernstoneDemo specific code is in that version. If you want the latest and greatest, get it from conerstoneTools.  Be aware that the tools library is under active development and bugs likely exist.

2) There have been some issues related to both IE and touch events but I don't know the current state of things - perhaps Erik Ziegler or Scott Poulin can chime in.  Search this forum for relevant threads

This is what I had figured as I have been using it as opposed to any other tools from the other Cornerstone repositories.I just wanted to confirm this to ensure that I was using the latest builds for the appropriate tools. I did come across what appears to be an issue, but it may just be an oversight on my part, which can be described in this issue on GitHub. Basically, some of the tools do not appear to work in Internet Explorer, when I attempt to build them on my own, however they seem to work just fine within an example like the All Image Tools example, but not in the CornerstoneDemo example (or my own similar implementation). This seems to be limited to the Angle and other measurement tools (e.g. Rectangular ROI, Length, etc.).

It appears this same issue was addressed in this discussion, however I suppose it wouldn't hurt to see if anyone has encountered a working approach.

3) Touch has only been implemented for a few tools - nobody is working on adding touch to all of them AFAIK.  Contributions are welcome :)

I suppose there are only really a few that touch would be needed for. Perhaps once I get a few of the other issues hashed out, I'll consider delving into that as one of the major benefits of a viewer such as this is that it would work really well on tablet and other touch-sensitive devices.

4) I believe that one of cornerstone's key strengths when compared to other open source image viewers is that it restricts itself to the viewport level.  Layouts are actually pretty easy to do with HTML - simply create a DIV for each viewport and use CSS to size them properly.  Cornerstone does have support for multi viewport functionality such as reference lines and stack synchronization and there examples of both in the cornerstoneTools library.

Thanks again Chris. I had noticed that there were some very rough plans in progress to handle this in the CornerstoneDemo, but I noticed that never really came to fruition. I'll start trying to piece together some proof of concepts with regards to handling this when I get a chance to hopefully flesh out that same logic.

[Erik Ziegler]

Touch support actually works for quite a few of the tools. I have run into some cases where the mouseInput and touchInput seems to have overlapped, though, causing events to fire more than once when both mouse and touch versions of the same tool were activated. I don't have a good solution for this yet, since I don't know why the touch events were even firing in the first place.

I have just got HammerJS 2.0.4 working with cornerstoneTools, so that should allow us to support more devices. The PR is here:

https://github.com/chafey/cornerstoneTools/pull/56

and a cleaned up demo of all the touch tools is here:

https://rawgit.com/swederik/cornerstoneTools/hammerjs/examples/alltouchtools/index.html

Some improvements may be necessary to make it easier to touch the tools, though (e.g. bigger handle touch areas).

[rionm...@gmail.com]

Hi Erik,

Those examples actually look great and required a bit of tinkering to get to work as expected. I came up with the following questions / notes regarding the changes as well.

rotateTouch missing a deactivate function

 

When debugging around and testing them out, I did notice that the highlighted line below continued to blow up and would cause the tool selection to not work as expected :

// helper function used by the tool button handlers to disable the active tool

// before making a new tool active

function disableAllTools()

{

            cornerstoneTools.panTouchDrag.deactivate(element);

            cornerstoneTools.rotateTouchDrag.deactivate(element);

            //cornerstoneTools.rotateTouch.deactivate(element);

            cornerstoneTools.ellipticalRoiTouch.deactivate(element);

            cornerstoneTools.angleTouch.deactivate(element);

            cornerstoneTools.rectangleRoiTouch.deactivate(element);

            cornerstoneTools.lengthTouch.deactivate(element);

            cornerstoneTools.probeTouch.deactivate(element);

            cornerstoneTools.zoomTouchDrag.deactivate(element);

            cornerstoneTools.wwwcTouchDrag.deactivate(element);

}

Commenting it out however appeared to resolve the issue, however it doesn't look like this same event has a deactivate function defined within the rotateTouch section in the CornerstoneTools.js file. There is a disable function, which can be seen below :

function disable(element) {

        $(element).off("CornerstoneToolsTouchRotate", touchRotateCallback);

}

I'm not sure if there is just a deactivate function missing or if the actual disable function is the one that needs to be called when disabling the tools.

Internet Explorer Touch Events 

I noticed I was able to use the same example that you provided on a desktop with Google Chrome, however when I attempted to switch over to something like Internet Explorer, it didn't appear to work as expected. I wasn't sure if this was a shortcoming of hammer.js or if the functionality simply isn't there at this point (e.g. events need to get wired up, etc.). I'm going to play around with your existing PR for a bit hopefully over the weekend to see if I can make any progress on that front, and if you know of anything else I should watch out for or have any input on the above issues, let me know.

Thanks again Eric,

Rion

[Erik Ziegler]

Hello!

Ah yeah sorry that was a copy/paste mistake. The rotateTouch tool doesn't actually have a deactivate function, as you noticed. I just changed the example to use disable instead. This is something we need to sort out for tools in which there is no difference between disable and deactivate. For the moment I'm just using disable.

As for IE Touch Events, I'm not really sure I understand. Are you using a touchscreen computer or something? What do you mean, exactly? If it's just a question of whether or not you can interact with the touch demos using internet explorer (with a mouse) on a desktop, it may be because at the moment we use the HammerJS Touch Emulator to allow testing via mouse clicks.

There is a report of the touch emulator not working on IE 10 already, so that could be what you're experiencing. See:

https://github.com/hammerjs/touchemulator/issues/1

Hope that helps,

Erik

[rionm...@gmail.com]

Hi again Eric,

Thanks for the clarification on the activate / deactivate issue, I figured that it was it was probably something minor like that. With regards to the IE Touch events, I was testing out your recent example on a desktop machine using the touch emulation (via a mouse) and the example appeared to work as expected within Google Chrome, however Internet Explorer wouldn't pick up any of the events.

After delving into this a bit, I determined than an exception was being triggered within the onTouch event in the touchInput.js section (cornerstoneTools.js; Line 310) which was passing in the actual Window object as an element as opposed to the proper container for the <canvas> element. 

By changing the following code from :

function onTouch(e) {

        var element = e.srcEvent.currentTarget,

            event,

            eventType;

        // Omitted for brevity
}

to :

function onTouch(e) {

        // FIX: Adjusted to handle targeting the appropriate node (previously the Window object was being targeted, resulting in a "element not enabled" issue

        var element = e.target.parentNode,

            event,

            eventType;

        // Omitted for brevity
}

This appears to have resolved the issue in Internet Explorer, which apparently enjoyed targeting the Window element. After making these changes, I performed some brief testing on both my local development machine (in Internet Explorer and Chrome) as well as on my Surface Pro 3 (running Windows 10 with Internet Explorer, Chrome and Edge with touch events). Everything appeared to work as expected using mouse-events and touch events for all of the major tools within your example :)

I thought about making a pull request, but I figured that there were so many and that the change was so minor that it would be easier for you to add it in.

[rionm...@gmail.com]

Additionally, you can see an example with these changes below as well if anyone else is interested in testing them out in the mean time.

https://rawgit.com/Rionmonster/cornerstone-local-testing/master/Examples/Index.html

[Erik Ziegler]

Great, thanks a lot. I committed the fix. Good to know it works on the Surface Pro as well, I've only been testing the touch events on an android phone.

[rionm...@gmail.com]

That's great to hear.

I've tested it on a few iOS devices that were lying around (e.g. iPhone, iPad) and I'm in the process of getting a colleague to try it on his Windows Phone. If those work properly, that should cover a huge percentage of users and devices.