Omeka 2.2 sneak preview: Derviative (thumbnail) changes, Imagick, and more
246 views
Skip to first unread message
John Flatness
Apr 3, 2014, 11:24:10 AM4/3/14
to omek...@googlegroups.com
We've been working for a little while here on Omeka 2.2, our next
feature release, and wanted to share a little bit of what's in the
pipeline. We have several major changes for derivative (aka thumbnail)
creation in the works. You can check them out on the Github branch
"derivatives" (https://github.com/omeka/Omeka/tree/derivatives), and
I'll also call out some of the more interesting changes here.
First, there's a brand new system in place to let an Omeka site
administrator change the way that derivative images get generated.
Previous versions of Omeka had one big class for creating thumbnails,
which incorporated lots of assumptions and specific code for handling
the ImageMagick command line tools.
In Omeka 2.2, there's a separation in place. The new derivative creator
class is generic, and only handles things that are always applicable to
our generation process. It passes off the actual creation of the new
thumbnail images to a derivative "strategy" class. All the code specific
to running the ImageMagick command line tools is refactored out into the
new ExternalImageMagick strategy class. At the same time, we dropped the
call to "identify" that we had added to the creation process. It didn't
serve a particularly useful purpose, and dropping it can cut the time
for making some thumbnails in half.
A site administrator can change the strategy they're using by just
specifying a class name in the application/config/config.ini file. This
is similar to how different Storage adapters can be selected there. In
the same way, entirely new strategies can be included in plugins and
selected in config.ini. A new strategy simply has to implement a single
method that creates one derivative image. We pass along the configured
pixel size constraint, the type of the image (fullsize, thumbnail,
etc.), and the mime type of the original file, to let strategies vary
their behavior accordingly.
These changes make it much easier to introduce the next major change:
Imagick extension support. In addition to just refactoring the old
derivative creator into a strategy, we've created a new strategy that
uses Imagick instead of calling out to the shell and the "convert" binary.
There are a few smaller changes rolled in here as well. We've added
configurable blacklist/whitelist support for mime types, so people with
installations that have trouble with a particular type can blacklist it,
or admins can be generous in the file types they allow to be uploaded,
but set a restrictive whitelist on which types will be processed for
thumbnails.
Finally, the new strategy classes have a system for accepting "options"
specified in the config.ini file. The idea here is to account for
situations where people want to make relatively minor changes to, say,
the command line we give to ImageMagick. As with most of these changes,
the focus is on adding flexibility without making people hack on the
Omeka core code.
This message grew a little longer than I was intially planning, so
here's a short recap. In Omeka 2.2, you can: swap out the derivative
creator code, use Imagick to create your thumbnails, blacklist or
whitelist particular file types from getting thumbnails, and set options
for configuring the thumbnail creator of your choice. These changes
aren't completely finished, and aren't set in stone. We've love to hear
any feedback about these features, the code itself, or things like
specific options that you'd like to see to make your specific use case
work better.
-John
feature release, and wanted to share a little bit of what's in the
pipeline. We have several major changes for derivative (aka thumbnail)
creation in the works. You can check them out on the Github branch
"derivatives" (https://github.com/omeka/Omeka/tree/derivatives), and
I'll also call out some of the more interesting changes here.
First, there's a brand new system in place to let an Omeka site
administrator change the way that derivative images get generated.
Previous versions of Omeka had one big class for creating thumbnails,
which incorporated lots of assumptions and specific code for handling
the ImageMagick command line tools.
In Omeka 2.2, there's a separation in place. The new derivative creator
class is generic, and only handles things that are always applicable to
our generation process. It passes off the actual creation of the new
thumbnail images to a derivative "strategy" class. All the code specific
to running the ImageMagick command line tools is refactored out into the
new ExternalImageMagick strategy class. At the same time, we dropped the
call to "identify" that we had added to the creation process. It didn't
serve a particularly useful purpose, and dropping it can cut the time
for making some thumbnails in half.
A site administrator can change the strategy they're using by just
specifying a class name in the application/config/config.ini file. This
is similar to how different Storage adapters can be selected there. In
the same way, entirely new strategies can be included in plugins and
selected in config.ini. A new strategy simply has to implement a single
method that creates one derivative image. We pass along the configured
pixel size constraint, the type of the image (fullsize, thumbnail,
etc.), and the mime type of the original file, to let strategies vary
their behavior accordingly.
These changes make it much easier to introduce the next major change:
Imagick extension support. In addition to just refactoring the old
derivative creator into a strategy, we've created a new strategy that
uses Imagick instead of calling out to the shell and the "convert" binary.
There are a few smaller changes rolled in here as well. We've added
configurable blacklist/whitelist support for mime types, so people with
installations that have trouble with a particular type can blacklist it,
or admins can be generous in the file types they allow to be uploaded,
but set a restrictive whitelist on which types will be processed for
thumbnails.
Finally, the new strategy classes have a system for accepting "options"
specified in the config.ini file. The idea here is to account for
situations where people want to make relatively minor changes to, say,
the command line we give to ImageMagick. As with most of these changes,
the focus is on adding flexibility without making people hack on the
Omeka core code.
This message grew a little longer than I was intially planning, so
here's a short recap. In Omeka 2.2, you can: swap out the derivative
creator code, use Imagick to create your thumbnails, blacklist or
whitelist particular file types from getting thumbnails, and set options
for configuring the thumbnail creator of your choice. These changes
aren't completely finished, and aren't set in stone. We've love to hear
any feedback about these features, the code itself, or things like
specific options that you'd like to see to make your specific use case
work better.
-John
Padraic Stack
Apr 3, 2014, 11:31:43 AM4/3/14
to omek...@googlegroups.com
Hi John, all,
> We've
> love to hear any feedback about these features, the code itself, or
> things like specific options that you'd like to see to make your
> specific use case work better.
I'm not sure if this is the sort of feedback you're looking for but it
is thumbnail related.
At the moment some of the square thumbnails created for the Gallery
section of Exhibit pages can look distorted or out of focus. See for
example
http://deevy.nuim.ie/exhibits/show/katieroche/katie-roche---abbeydebut
The same thing doesn't happen for the non-square thumbnails.
Regards and thanks,
--
Padraic
Padraic Stack | Digital Humanities Support Officer | NUI Maynooth |
padrai...@nuim.ie |Phone: Mon: 01 474 7187 Tue - Fri: 01 474 7197
> We've
> love to hear any feedback about these features, the code itself, or
> things like specific options that you'd like to see to make your
> specific use case work better.
is thumbnail related.
At the moment some of the square thumbnails created for the Gallery
section of Exhibit pages can look distorted or out of focus. See for
example
http://deevy.nuim.ie/exhibits/show/katieroche/katie-roche---abbeydebut
The same thing doesn't happen for the non-square thumbnails.
Regards and thanks,
--
Padraic
Padraic Stack | Digital Humanities Support Officer | NUI Maynooth |
padrai...@nuim.ie |Phone: Mon: 01 474 7187 Tue - Fri: 01 474 7197
Sylvain Machefert
Apr 3, 2014, 11:35:32 AM4/3/14
to omek...@googlegroups.com
Hi John,
one use case I've came across is the need to prevent the thumbnail generation on the fly (adding 100's jpg using Dropbox can provoke timeout calls). To correct this, I run https://omeka.org/codex/Plugins/DerivativeImages at night, as a background process, and have disabled the thumbnail generation by commenting the call to createDerivatives() in models/Job/FileProcessUpload.php. If it should be an option in a config file it would be easier (and if there's another way to easily manage that use case, I'd be happy to know it).
Sylvain
-John
--
You received this message because you are subscribed to the Google Groups "Omeka Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to omeka-dev+unsubscribe@googlegroups.com.
To post to this group, send email to omek...@googlegroups.com.
Visit this group at http://groups.google.com/group/omeka-dev.
For more options, visit https://groups.google.com/d/optout.
Rachel Donahue
Apr 3, 2014, 12:12:55 PM4/3/14
to omek...@googlegroups.com
Wow, this is all very exciting! One addition I'd like is to be able to create new derivative images for a range of item IDs or a specific tag. Though maybe this is more in the Derivative Images plug-in category?
Sometimes we wind up with a blank (or just dull) first page on a few items; it would be nice to be able to re-run thumbnail creation for just those items. My current plan is just to temporarily change IM's target page number and re-import those items with CSVImport.
-Rachel
Sometimes we wind up with a blank (or just dull) first page on a few items; it would be nice to be able to re-run thumbnail creation for just those items. My current plan is just to temporarily change IM's target page number and re-import those items with CSVImport.
-Rachel
John Flatness
Apr 3, 2014, 12:17:59 PM4/3/14
to omek...@googlegroups.com
Actually it turns out this is pretty directly related to the stuff I was
mentioning.
I can confirm your blurriness problem on the "stock" derivative creator.
Your problem is with the square thumbnails themselves, not the Exhibit
Builder (it just happens to be using those thumbnails).
The issue is that for ImageMagick, we use what's essentially a
workaround to create square thumbnails. Newer versions have built-in
functionality to do the same thing, but out of concern for compatibility
we don't use them. The "workaround" requires us to make an assumption
about which dimension of an image will be the longer one. Generally this
isn't a problem, but when an image is much taller than it is wide, like
your pictures of newspaper columns, you get very poor results like that.
I tried out your image on the new Imagick strategy, and it's got the
kind of sharpness you would expect. Imagick provides a method that's
basically purpose-built for creating things like our square thumbnails,
and it handles the "very tall" images nicely.
The new derivative changes would provide a few options. First, if you
have it available, you could use Imagick, which should provide you with
good results. Second, we could enable the "new" ImageMagick command-line
support for square thumbnails, but have an option for enabling it, so we
can both get the new feature and preserve compatibility.
-John
mentioning.
I can confirm your blurriness problem on the "stock" derivative creator.
Your problem is with the square thumbnails themselves, not the Exhibit
Builder (it just happens to be using those thumbnails).
The issue is that for ImageMagick, we use what's essentially a
workaround to create square thumbnails. Newer versions have built-in
functionality to do the same thing, but out of concern for compatibility
we don't use them. The "workaround" requires us to make an assumption
about which dimension of an image will be the longer one. Generally this
isn't a problem, but when an image is much taller than it is wide, like
your pictures of newspaper columns, you get very poor results like that.
I tried out your image on the new Imagick strategy, and it's got the
kind of sharpness you would expect. Imagick provides a method that's
basically purpose-built for creating things like our square thumbnails,
and it handles the "very tall" images nicely.
The new derivative changes would provide a few options. First, if you
have it available, you could use Imagick, which should provide you with
good results. Second, we could enable the "new" ImageMagick command-line
support for square thumbnails, but have an option for enabling it, so we
can both get the new feature and preserve compatibility.
-John
John Flatness
Apr 3, 2014, 12:21:56 PM4/3/14
to omek...@googlegroups.com
The target page number is one of those things that I think is pretty
likely to end up as an option that can be configured.
The mime-type and Imagick support also offers us a few other interesting
pathways here. With Imagick we should be able to tell it to detect how
many "pages" are actually there and act differently depending on their
number, or specify different behavior for, say, PDF files, or video files.
For us, the particularly interesting facet is that people who have
specific needs could extend our derivative strategies, or make their own
from whole cloth, and add some of these specific behaviors that make
sense for their particular use cases, without us necessarily having to
provide explicit options for everything.
As for "re-deriving," that does seem like something that's more geared
as a request for the Derivative Images plugin than the core feature per se.
Thanks for the feedback!
-John
On 04/03/2014 12:12 PM, Rachel Donahue wrote:
> Wow, this is all very exciting! One addition I'd like is to be able to
> create new derivative images for a range of item IDs or a specific tag.
> Though maybe this is more in the Derivative Images plug-in category?
>
> Sometimes we wind up with a blank (or just dull) first page on a few
> items; it would be nice to be able to re-run thumbnail creation for just
> those items. My current plan is just to temporarily change IM's target
> page number and re-import those items with CSVImport.
>
> -Rachel
>
> On Thursday, April 3, 2014 11:24:10 AM UTC-4, John Flatness wrote:
>
> We've been working for a little while here on Omeka 2.2, our next
> feature release, and wanted to share a little bit of what's in the
> pipeline. We have several major changes for derivative (aka thumbnail)
> creation in the works. You can check them out on the Github branch
> "derivatives" (https://github.com/omeka/Omeka/tree/derivatives
> <https://github.com/omeka/Omeka/tree/derivatives>), and
> <mailto:omeka-dev+...@googlegroups.com>.
likely to end up as an option that can be configured.
The mime-type and Imagick support also offers us a few other interesting
pathways here. With Imagick we should be able to tell it to detect how
many "pages" are actually there and act differently depending on their
number, or specify different behavior for, say, PDF files, or video files.
For us, the particularly interesting facet is that people who have
specific needs could extend our derivative strategies, or make their own
from whole cloth, and add some of these specific behaviors that make
sense for their particular use cases, without us necessarily having to
provide explicit options for everything.
As for "re-deriving," that does seem like something that's more geared
as a request for the Derivative Images plugin than the core feature per se.
Thanks for the feedback!
-John
On 04/03/2014 12:12 PM, Rachel Donahue wrote:
> Wow, this is all very exciting! One addition I'd like is to be able to
> create new derivative images for a range of item IDs or a specific tag.
> Though maybe this is more in the Derivative Images plug-in category?
>
> Sometimes we wind up with a blank (or just dull) first page on a few
> items; it would be nice to be able to re-run thumbnail creation for just
> those items. My current plan is just to temporarily change IM's target
> page number and re-import those items with CSVImport.
>
> -Rachel
>
> On Thursday, April 3, 2014 11:24:10 AM UTC-4, John Flatness wrote:
>
> We've been working for a little while here on Omeka 2.2, our next
> feature release, and wanted to share a little bit of what's in the
> pipeline. We have several major changes for derivative (aka thumbnail)
> creation in the works. You can check them out on the Github branch
> "derivatives" (https://github.com/omeka/Omeka/tree/derivatives
> --
> You received this message because you are subscribed to the Google
> Groups "Omeka Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to omeka-dev+...@googlegroups.com
> You received this message because you are subscribed to the Google
> Groups "Omeka Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:omeka-dev+...@googlegroups.com>.
> To post to this group, send email to omek...@googlegroups.com
> <mailto:omek...@googlegroups.com>.
Anurag Spatzenegger
Apr 3, 2014, 12:28:37 PM4/3/14
to omek...@googlegroups.com
I wholeheartedly welcome this new behavior :)
John Flatness
Apr 3, 2014, 6:32:16 PM4/3/14
to omek...@googlegroups.com
Well, setting a "whitelist" that was empty would effectively disable the
derivative creator under this system, and you set that from the
config.ini. However, you'd have to go back and remove that config
setting before you ran DerivativeImages, since otherwise it would apply
the same whitelist, and not make any thumbnails.
-John
On 04/03/2014 11:35 AM, Sylvain Machefert wrote:
> Hi John,
> one use case I've came across is the need to prevent the thumbnail
> generation on the fly (adding 100's jpg using Dropbox can provoke
> timeout calls). To correct this, I run
> https://omeka.org/codex/Plugins/DerivativeImages at night, as a
> background process, and have disabled the thumbnail generation by
> commenting the call to createDerivatives() in
> models/Job/FileProcessUpload.php. If it should be an option in a config
> file it would be easier (and if there's another way to easily manage
> that use case, I'd be happy to know it).
>
> Sylvain
>
>
>
> 2014-04-03 17:24 GMT+02:00 John Flatness <jo...@zerocrates.org
> <mailto:jo...@zerocrates.org>>:
> <https://github.com/omeka/Omeka/tree/derivatives>), and I'll also
> send an email to omeka-dev+unsubscribe@__googlegroups.com
> <mailto:omeka-dev%2Bunsu...@googlegroups.com>.
> Visit this group at http://groups.google.com/__group/omeka-dev
> <http://groups.google.com/group/omeka-dev>.
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
> <mailto:omeka-dev+...@googlegroups.com>.
derivative creator under this system, and you set that from the
config.ini. However, you'd have to go back and remove that config
setting before you ran DerivativeImages, since otherwise it would apply
the same whitelist, and not make any thumbnails.
-John
On 04/03/2014 11:35 AM, Sylvain Machefert wrote:
> Hi John,
> one use case I've came across is the need to prevent the thumbnail
> generation on the fly (adding 100's jpg using Dropbox can provoke
> timeout calls). To correct this, I run
> https://omeka.org/codex/Plugins/DerivativeImages at night, as a
> background process, and have disabled the thumbnail generation by
> commenting the call to createDerivatives() in
> models/Job/FileProcessUpload.php. If it should be an option in a config
> file it would be easier (and if there's another way to easily manage
> that use case, I'd be happy to know it).
>
> Sylvain
>
>
>
> 2014-04-03 17:24 GMT+02:00 John Flatness <jo...@zerocrates.org
>
> We've been working for a little while here on Omeka 2.2, our next
> feature release, and wanted to share a little bit of what's in the
> pipeline. We have several major changes for derivative (aka
> thumbnail) creation in the works. You can check them out on the
> Github branch "derivatives"
> (https://github.com/omeka/__Omeka/tree/derivatives
> We've been working for a little while here on Omeka 2.2, our next
> feature release, and wanted to share a little bit of what's in the
> pipeline. We have several major changes for derivative (aka
> thumbnail) creation in the works. You can check them out on the
> Github branch "derivatives"
> <https://github.com/omeka/Omeka/tree/derivatives>), and I'll also
> <mailto:omeka-dev%2Bunsu...@googlegroups.com>.
> To post to this group, send email to omek...@googlegroups.com
> <mailto:omek...@googlegroups.com>.
> Visit this group at http://groups.google.com/__group/omeka-dev
> <http://groups.google.com/group/omeka-dev>.
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Omeka Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to omeka-dev+...@googlegroups.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Omeka Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:omeka-dev+...@googlegroups.com>.
> To post to this group, send email to omek...@googlegroups.com
> <mailto:omek...@googlegroups.com>.
Rachel Donahue
Apr 4, 2014, 12:59:24 PM4/4/14
to omek...@googlegroups.com
So glad to hear that! I Many institutions include disclaimer pages in their PDFs, and identical thumbnails != useful.
Yes, I've seen those granular file-type preferences in IM. For us, it would definitely be handy to create a strategy where PDFs pull thumbs from page 2 (1, to IM), while JPGs use page 1 (the only page).
Now what are the chances of this being ready by the end of next week when I need to re-import all of my content? ;)
-Rachel
John Flatness
Apr 4, 2014, 5:10:00 PM4/4/14
to omek...@googlegroups.com
A further follow-up on the "tall images = crappy square thumbnails" issue:
The flag necessary to make the square thumbnails sharper was added in
ImageMagick 6.3.8-2, from January 2008. I'm open to the argument that
there's not a genuine compatibility problem here, but it would greatly
depend on the versions of ImageMagick that people have available to them
out in the wild.
-John
The flag necessary to make the square thumbnails sharper was added in
ImageMagick 6.3.8-2, from January 2008. I'm open to the argument that
there's not a genuine compatibility problem here, but it would greatly
depend on the versions of ImageMagick that people have available to them
out in the wild.
-John
Reply all
Reply to author
Forward
0 new messages