Advice on basic operations: zoom, crop and splice

[Adam Hughes]

Hi everyone,

I'm used to working with big images of lots of particles.  In the notebook, I'd like to be able to look at a full image, and then at a zoomed in region of interest.  A few basic questions come to mind:

    

 Is there a zoom/crop function or preffered approach to basic manipulations of zooming and cropping, or would I have to do this at the numpy or matplotlib level?  I saw that there's a rectangle function that probably would be useful here.  Does anyone have any examples or personal code built for doing some of these common manipulations?   Ideally, I want to take the fastest approach to:

1.  Selecting a rectangular region of interest (ROI).

2.  Cropping or zooming in on this region, and storing the ROI as its own array/image.

3.  If possible, removing the ROI from the original image, and splicing the original image back together.  If this is possible, that would be amazing.  This would allow us to effectively cut out regions of our images that are obviously contaminates. 

Thanks.

[Josh Warner]

The backend functionality for all of this exists. I believe all of this can be accomplished with masks and NumPy array slicing.

The problem is that generally speaking, these are not interactive or GUI level operations. For example, if you want a zoomed-in ROI you can slice that region out and then `plt.imshow(sliced_img)`, but this requires manually entering the ROI corners in the slicing operation.

`skimage.viewer` includes a method to select and save rectangular regions interactively, and could probably be extended to allow clipping them out as well. However, the viewer requires PyQt or PySide and cannot be embedded in an iPython notebook.

Are you specifically looking for iPython notebook capable solutions? Or would a tweaked Viewer plugin be sufficient?

[Chintak Sheth]

Hi Adam

On Nov 23, 2013 12:33 AM, "Adam Hughes" <hughes...@gmail.com> wrote:
>
> 3.  If possible, removing the ROI from the original image, and splicing the original image back together.  If this is possible, that would be amazing.  This would allow us to effectively cut out regions of our images that are obviously contaminates. 

Yes, this is possible. It is also known as Image Inpainting. There are two algorithm implementations for the same, although they haven't yet been merged.

Fast Marching Algorithm: (Fast, introduce some blurriness)

Here is the dev branch: https://github.com/chintak/scikit-image/tree/inpaint

PR for it: https://github.com/scikit-image/scikit-image/pull/663

Exemplar based Inpainting: (Slower, very accurate for repeating patterns, reconstructs using sample image patches from the image)

Dev branch: https://github.com/chintak/scikit-image/tree/inpaint_exemplar

PR: https://github.com/scikit-image/scikit-image/pull/706

For the first one, a skimage.viewer plugin is also there: ``viewer_examples/plugins/demo_inpaint.py``

Regarding your first 2 points, Josh shed some light on how you can do this using `skim age.viewer`. An interesting thread regarding interactive image manipulation in iPython notebook came up lately which might interest you: https://groups.google.com/forum/#!topic/scikit-image/vrPzanJ42rs. I'm not sure if you can select an ROI in the image, but that'd be worth checking out. If we can't directly select a rectangular region then probably defining 4 sliders in the widget: left, right, top, bottom, and moving them around to select the ROI.

Chintak

[Stéfan van der Walt]

On Sat, Nov 23, 2013 at 8:16 AM, Chintak Sheth <chinta...@gmail.com> wrote:
> An interesting thread regarding interactive image
> manipulation in iPython notebook came up lately which might interest you:
> https://groups.google.com/forum/#!topic/scikit-image/vrPzanJ42rs.

I spoke to Brian Granger about the possibility of building the viewer
described above in IPython. Here is his response:

"""
Definitely possible - shouldn't be too bad. Here is a rough sketch:

* You can define a Python object with traitlets.
* You get a JavaScript object that has the same exact attributes as
the Python object - we keep those attributes completely synchronized
on the two sides automatically. When the JavaScript side (through UI
control changes) changes the attributes we sync the python side and
all of the traitlets callbacks will fire. When the Python side makes
changes, the JS side is updated and the UI will refresh. The time
scale for all of this is super fast - at least running on localhost.
* You can then create arbitrary "view" code for the JavaScript side.
You get a div to put the UI in and can do whatever you want.
* The JavaScript and Python objects also have methods for sending
arbitrary JSON messages at any time, so you can stream data, etc.

To get familiar with how it all works, have a look at the notebooks in
this branch:

https://github.com/jdfreder/ipython/tree/widget-msg/examples/widgets

They show how to develop new widgets from scratch as well as how all
of the existing ones can be used.
"""

I am really keen to try this out next week, although I may have to up
my Javascript-foo a bit.

Stéfan

[Adam Hughes]

Thanks for all of your help guys.  I was unaware of image Inpainting, so thank you Chintak for bringing that fascinating concept to my attention.

Josh, thanks for your help.  I am capable enough to do such manipulations in numpy and matplotlib; hell,I could even just save a zoomed/crop version of the images of interest.  I was certainly interested in doing such manipulations in the notebook.  I wouldn't worry about tweaking the viewer, as it would just be easier for me to, as I said, do the manipulations in and save them to separate images.

The discussion that followed was quite interesting, and in line with what I expected might be the case.  It's really great to see the progress of iPython widgets.  I'm fairly competent at traits/chaco, but don't have a javascript background.  Stefan, any idea how much javascript competency one might need to begin to approach this effort?

[Adam Hughes]

I made a couple basic utilties:

  zoom, crop, and zoomplot

They're very straightforward, but there was just enough nuance involved that working directly with numpy and matplotlib would become a pain.

See the attached notebook and/or pdf.  The PDF was generaged with nbconvert, and the image quality is poor.  Feel free to use and distribute this; please the ignore the content claim on the title page; it defaults to all of our pdfs that we generate with nbconvert.

On Friday, November 22, 2013 2:03:04 PM UTC-5, Adam Hughes wrote:

[Adam Hughes]

Sorry, last post.  Here's a slightly more complete version for sake of archiving:

--
You received this message because you are subscribed to a topic in the Google Groups "scikit-image" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scikit-image/Ke7_TwUew4U/unsubscribe.
To unsubscribe from this group and all of its topics, send an email to scikit-image...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Michael Aye]

Hey Adam, your Notebook PDF output is *sick*! ;)

Care to share how you set it up so nicely?

Michael

[Adam Hughes]

Thanks!  Let me make a few adaptaions and host it proper in a day or two

[Stéfan van der Walt]

Hi Adam

On Tue, Nov 26, 2013 at 4:45 AM, Adam Hughes <hughes...@gmail.com> wrote:
> I made a couple basic utilties:
> zoom, crop, and zoomplot

Thanks for sharing!

A tiny IPython tip: if you end the last line of your cell with a
semi-colon, the printing of the resulting object is disabled. Very
useful for doing plots.

Regards
Stéfan

[Adam Hughes]

Thanks Stefan, I didn't know that.  Appreciate it.