Imagemagick Vs Libvips

[Jeremias Resendez]

Ithas around 300 operations coveringarithmetic, histograms, convolution, morphological operations, frequencyfiltering, colour, resampling, statistics and others. It supports a largerange of numeric formats,from 8-bit int to 128-bit complex. Images can have any number of bands.It supports a good range of image formats, including JPEG, TIFF, PNG, WebP, FITS, Matlab, OpenEXR, PDF, SVG, HDR, PPM, CSV, GIF, Analyze,NIfTI, DeepZoom, and OpenSlide. It can also load images via ImageMagick orGraphicsMagick, letting it load formats like DICOM.

It comes with bindings for C,C++,and the command-line. Full bindingsare available for Ruby,Python,PHP, .NET, Go, andLua. libvipsis used as an image processing engine by sharp (onnode.js),bimg,sharp for Go,Ruby on Rails,carrierwave-vips,mediawiki,PhotoFlow and others.The official libvips GUI is nip2,a strange combination of a spreadsheet and an photo editor.

I'm working on building a website with Gatsby and whenever I try to build I get this error. I have installed libvips with Homebrew and it automatically includes ImageMagick. I then deleted sharp from node_modules and reinstalled sharp, but I still get this error. I'm new to doing anything on the command line so any help is appreciated!

It is my experience, running Active Storage in production since version 5.2, that this is not a good enough default and Rails should steer users away from it, and incentivize them to use vips instead.

Back when we were in Heroku this got so bad that we could tell when a user had uploaded a large amount of photos at the same time by the fact that there was a spike in queue times and the auto scaler had kicked in.

ImageMagick is very memory intensive when processing variants. Benchmarks by libvips author demonstrate it using 3GB memory versus 200MB by vips. With cameras in mobile phones getting better and better, the photos that must be processed are becoming larger. This inefficiency causes problems for developers running their apps in servers with tight memory constraints, like 512MB or 1GB (eg: Heroku). With Rails apps already being memory intensive, they should be always close to the memory ceiling.

I agree about making vips the default image processor and I think there is another argument to be made: security. We are not running ActiveStorage due to the high security we need for our storage. Running transformations on uploaded files is often a security risk as the tools have lots of vulnerabilities. Most developers seem to run the transformation binaries on the same webserver as the rest of their application, making it even more insecure.

However, libvips as usually distributed by Debian etc. includes many, many other loaders for more obscure (and much less well tested) formats like NIfTI and FITS, plus a complete copy of imagemagick as a fallback processor for unsupported formats, like BMP and ICO.

If people use the Debian binary directly, they are unlikely to see a significant improvement in security. I think I would include a note in the docs recommending the use of a stripped-down libvips which only included support for the popular and well-tested web formats.

Ideally I suppose Heroku would make a libvips-web package with just the fuzzed loaders and include it in their buildpack. The libvips team would be delighted to help in any way we can with setting this up.

Heroku unfortunately includes imagemagick as part of the base image for all their stacks and when I asked about vips they said it was a frequent request, and they had considered it, but adding extra packages would make deploys slower. So we can deduce that at the time they had no intention to remove magick

Currently that process is failing because AS employs delayed/lazy generation, and therefore has to Marshal.dump all parameters that will be passed to vips. And according to Janko, Vips::Image is failing marshalling because it employs C Structs that ruby does not know how to Marshal.

Correct. Active Storage assumes that all transformations can be encoded as strings and that one machine will generate said string, and another machine will take said string and use to the process the image

It would be reasonable to assume that the first, time this variant of the original avatar file is requested, AS would download the original file, apply the transformations, upload it back storage and the url for the resized image. That is not actually happens however. This is what you get:

This is the URL of a special AS controller that is responsible for serving variants of images it has stored. signed_id is the encrypted ID that AS uses to locate the original file, and variation_key is the result of this:

And Active Storage will handle downloading, processing, uploading and tracking each of those variants without forcing me to add extra columns to my models to store the urls. But to do this, it needs to be able to take a set of transformations and turn it into a unique key that will always be the same for a given set of transformations:

image_processing would then need a new backend that generated RVL instead of calling ruby-vips directly, plus a thing to serialise into string, and a method to execute a stored RVL program. #method_missing in image_processing would simply add a node to the RVL being generated.

That library is something that I was working on for fun, but as it started to reach a decently stable-ish point. The blocker for me moving forward was ensuring safety for the user as scaling LibVips in a massively concurrent manner can lead to crashes that trickle up to the BEAM.

However, there is elxvips Hex , it uses NIFs, I tried it on my testing machine and so far, it works well, image transformations are 3-4x faster compared to imagemagick. Does anyone have any experience with it in production?

vix tries to be complete binding for vips, supporting almost all operations and options. If your use case has complex image manipulation pipelines then you can do that efficiently with vix. afaik other libraries does not support all operations or pipelines. You can think of vix as an elixir binding library, which aims to be something like ruby-vips.

@akash-akya Wonderful work. You know the best part of the library? The two LiveBooks. I think we are in for some revolution in terms of documentation for elixir libraries.

Thanks for the good work.

Our base images include many widely used dependencies. For dependencies related to more specific workloads, I would suggest running your service in a Docker environment instead of our native Ruby environment.

Since Rails 7 is planning to use libvips as a default image-dealing library (and not imagemagick any longer), do you in Render team plan to include libvips to your libraries on a plain Ruby image? Thx!

I really think having our image processing handled by govips which wraps libvips could provide a lot of additional functionality and could potentially increase performance. Here are some tests to check out:

ImageMagick is often recommended for image optimization . This is a good tool, but the result of its work usually needs additional optimization, while other tools do better. We recommend libvips , although it is a lower-level tool and more demanding on the technical skills of the user. At ImageMagick also periodically found security vulnerabilities, which are desirable to know about.

Image blobs can have variants that are the result of a set of transformations applied to the original. These variants are used to create thumbnails, fixed-size avatars, or any other derivative image from the original.

Variants rely on ImageProcessing gem for the actual transformations of the file, so you must add gem "image_processing" to your Gemfile if you wish to use variants. By default, images will be processed with ImageMagick using the MiniMagick gem, but you can also switch to the libvips processor operated by the ruby-vips gem).

Note that to create a variant it's necessary to download the entire blob file from the service. Because of this process, you also want to be considerate about when the variant is actually processed. You shouldn't be processing variants inline in a template, for example. Delay the processing to an on-demand controller, like the one provided in ActiveStorage::RepresentationsController.

When you do want to actually produce the variant needed, call processed. This will check that the variant has already been processed and uploaded to the service, and, if so, just return that. Otherwise it will perform the transformations, upload the variant to the service, and return itself again. Example:

This will create and process a variant of the avatar blob that's constrained to a height and width of 100. Then it'll upload said variant to the service according to a derivative key of the blob and the transformations.

Returns the URL of the variant on the service. This URL is intended to be short-lived for security and not used directly with users. Instead, the service_url should only be exposed as a redirect from a stable, possibly authenticated URL. Hiding the service_url behind a redirect also gives you the power to change services without updating all URLs. And it allows permanent URLs that redirect to the service_url to be cached in the view.

GD introduces some complex repeating and alternating patterns on downscaled images. On many photos it was almost impossible to see the effect, but on some simple illustrations or images with slight gradients it would be very obvious. The artifacts introduced additional complexity in the images which would yield poor compression.

The below image groups demonstrates an example of the type of noise-artifact patterns that can be added to downscaled images when resizing with PHP+libgd. The rightmost image highlights the differences between the original and downscaled images. Click the thumbnails for larger versions.

The first image has a solid background that should be the same color for most of the image. You can see the noise patterns shift in the different encoding tiles. The second image has a more complex background and shows a more varied noise pattern.

3a8082e126