Download sizes for TIFF, PSD, etc.

[Stefan Wild]

Please correct me if I'm wrong, but in my out-of-the-box 1.5.274
installation TIFFs, PSDs and other ImageMagick-handled files are not
resized (or shown) at any downloadable size. This is why I made a few
modifications that add the following functionality:

- Resizing to all sizes of TIFF, PSD and EPS (and with two more
changes probably more formats) files
- Always resizing to "hpr" (High Resolution Print, i.e. original
resolution JPEG) size for these files
- Collection Download option "original file" which always downloads
the source file, while "high resolution print" will download the
original size JPEGs
- Add Resource ID as a prefix to filenames when downloading collection
ZIP using original filenames (to avoid filename conflicts that lead to
omission of one or more files)
- Flattening of multi-layer TIFFs or PSDs when rendering preview/
download sizes
- Strip ALL profiles if not configured to preserve them

These are the diffs for the affected files:

diff include/general.php include/general_orig.php
45c45
< if (($extension=="") || ($size !="")) {$extension="jpg";}
---
> if ($extension=="") {$extension="jpg";}
269,288d268
< # add the original image
< $return=array();
< $lastname="";$lastpreview=0;$lastrestricted=0;
< $path2=get_resource_path($ref,'',false,$extension);
< if (file_exists($path2))
< {
< $returnline=array();
< $returnline["name"]=$lastname;
< $returnline["allow_preview"]=$lastpreview;
< $returnline["allow_restricted"]=$lastrestricted;
< $returnline["path"]=$path2;
< $returnline["id"]="";
< if ((list($sw,$sh) = @getimagesize($path2))===false)
{$sw=0;$sh=0;}
< if (($filesize=filesize($path2))===false)
{$returnline["filesize"]="?";$returnline["filedown"]="?";}
< else {$returnline["filedown"]=ceil($filesize/50000) . " seconds @
broadband";$returnline["filesize"]=formatfilesize($filesize);}
< $returnline["width"]=$sw;
< $returnline["height"]=$sh;
< $returnline["extension"]=$extension;
< $return[]=$returnline;
< }
290a271,272
> $return=array();
> $outputfirst=false;$lastname="";$lastpreview=0;
292,293c274,275
< {
< $path=get_resource_path($ref,$sizes[$n]["id"],false);
---
> {
> $path=get_resource_path($ref,$sizes[$n]["id"],false,$extension);
295a278,298
> if ($outputfirst==false)
> {
> # add the original image
> $path2=get_resource_path($ref,'',false,$extension);
> if (file_exists($path2))
> {
> $returnline=array();
> $returnline["name"]=$lastname;
> $returnline["allow_preview"]=$lastpreview;
> $returnline["allow_restricted"]=$lastrestricted;
> $returnline["path"]=$path2;
> $returnline["id"]="";
> if ((list($sw,$sh) = @getimagesize($path2))===false) {$sw=0;$sh=0;}
> if (($filesize=filesize($path2))===false) {$returnline["filesize"]="?";$returnline["filedown"]="?";}
> else {$returnline["filedown"]=ceil($filesize/50000) . " seconds @ broadband";$returnline["filesize"]=formatfilesize($filesize);}
> $returnline["width"]=$sw;
> $returnline["height"]=$sh;
> $return[]=$returnline;
> $outputfirst=true;
> }
> }
310d312
< $returnline["extension"]='jpg';
981c983
< ?>
---
> ?>


diff include/image_processing.php include/image_processing_orig.php
231c231
< if (preg_match('/^(tif[f]?|jp[e]?g|psd|eps)$/i', $extension))
---
> if (($extension=="jpg") || ($extension=="jpeg") || ($extension=="png") || ($extension=="gif"))
233c233
< # Create image previews for selected file types (JPEG, TIFF, PSD,
EPS)
---
> # Create image previews for built-in supported file types only (JPEG, PNG, GIF)
239c239
< # fetch source image size, if we fail, exit this function (file
not an image, or file not a valid jpg/png/gif). This somehow works
even with TIFF and PSD!?
---
> # fetch source image size, if we fail, exit this function (file not an image, or file not a valid jpg/png/gif).
258c258
< if (($sw>$tw) || ($sh>$th) || ($id=="thm") || ($id=="col") ||
($id=="hpr"))
---
> if (($sw>$tw) || ($sh>$th) || ($id=="thm") || ($id=="col"))
263,265d262
< if ($id=="hpr") {
< $ratio = min($ratio, 1);
< }
269,272d265
< if (($ratio == 1) && (!preg_match('/^(tif[f]?|jp[e]?g|psd|eps)$/
i', $extension))) {
< break;
< }
<
287c280
< $profile="+profile \"*\" -colorspace RGB"; # By default, strip
the colour profiles ('+' is remove the profile, confusingly)
---
> $profile="+profile icc +profile xmp +profile exif +profile iptc -colorspace RGB"; # By default, strip the colour profiles ('+' is remove the profile, confusingly)
290c283
< $command2 = $command . " \"$file\"[0] -flatten $profile -
quality $imagemagick_quality -resize " . $tw . "x" . $th . " \"$path
\"";
---
> $command2 = $command . " \"$file\"[0] $profile -quality $imagemagick_quality -resize " . $tw . "x" . $th . " \"$path\"";


diff view.php view_orig.php
174c174
< if (($resource["has_image"]==1) && (preg_match('/^(tif[f]?|jp[e]?g|
psd|eps)$/i', $resource["file_extension"])))
---
> if (($resource["has_image"]==1) && (($resource["file_extension"]=="jpg") || ($resource["file_extension"]=="jpeg")))
209d208
< $headline = ($n == 0) ? strtoupper($resource["file_extension"]) .
" " . $lang["file"] : i18n_get_translated($sizes[$n]["name"]);
212c211
< <td><h2><?=$headline?></h2>
---
> <td><h2><?=i18n_get_translated($sizes[$n]["name"])?></h2>
218c217
< <a href="terms.php?ref=<?=$ref?>&k=<?=$k?>&url=<?
=urlencode("download_progress.php?ref=" . $ref . "&size=" . $sizes[$n]
["id"] . "&ext=" . $sizes[$n]["extension"] . "&k=" . $k)?>"><?=
$lang["download"]?></a>
---
> <a href="terms.php?ref=<?=$ref?>&k=<?=$k?>&url=<?=urlencode("download_progress.php?ref=" . $ref . "&size=" . $sizes[$n]["id"] . "&ext=" . $resource["file_extension"] . "&k=" . $k)?>"><?=$lang["download"]?></a>


diff collection_download.php collection_download_orig.php
12d11
< $submitted=getvalescaped("submitted","");
14c13
< if ($submitted != "")
---
> if ($size!="")
64,68c63,64
< $filename=get_resource_data($ref);
< $filename=$ref.'_'.$filename["file_path"];
< preg_match('/\.[^\.]+$/', $p, $pext);
< $filename=preg_replace('/\.[^\.]+$/', $pext[0], $filename);
<
---
> $filename=get_resource_data($ref);$filename=$filename["file_path"];
>
129d124
< $sizes[]=array("id" => "", "name" => "Original files");
135c130
< <div class="Inline"><input name="submitted" type="submit"
value="&nbsp;&nbsp;<?=$lang["download"]?>&nbsp;&nbsp;" /></div>
---
> <div class="Inline"><input name="Submit" type="submit" value="&nbsp;&nbsp;<?=$lang["download"]?>&nbsp;&nbsp;" /></div>



Please let me know if there is any other (better) way to submit
patches.

Best regards,
Stefan

[Stefan Wild]

Sorry, the line breaks that Google inserts break the diffs. Just let
me know where to send them...

[Stefan Wild]

I've uploaded a single patch file that should work from the RS
(1.5.274 release) home dir with the following command:

patch -p0 -i RS_20080826.patch

Please try first with the --dry-run option.

Patch download:
http://www.sw4.de/wp-content/uploads/2008/08/rs_20080826.patch

German blog post:
http://www.sw4.de/2008/08/tiff-psd-und-eps-mit-resourcespace-15/

Best regards,
Stefan

[Stefan Wild]

There was a "break" where there should be a "continue". Fixed this.
New download at:
http://www.sw4.de/wp-content/uploads/2008/08/rs_20080826.patch

[Stefan Wild]

New download at (sorry for the typo!):
http://www.sw4.de/wp-content/uploads/2008/08/rs_20080827.patch

[Stefan Wild]

An improved version of these changes will probably commited to SVN,
soon. I recommend waiting for that.

Best regards,
Stefan

[ronhof]

I'am very eager to have this committed.....

The logic would be to have any images produce thumbs and all other
predefined sizes. Regardless of format.

Will color profiles be stripped only form thumbs, not for downloadable
hi res jpgs.

-Ron

> > > > > me know where to send them...- Hide quoted text -
>
> - Show quoted text -

[Stefan Wild]

> The logic would be to have any images produce thumbs and all other
> predefined sizes. Regardless of format.

Correct. In the first step this will be dependent on the extension. I
have tested so far with TIFF, PSD and Photoshop EPS. Most other
formats that ImageMagick supports should work as well.

> Will color profiles be stripped only form thumbs, not for downloadable
> hi res jpgs.

If you set $imagemagick_preserve_profiles=true in your config.php,
then all profiles that are compatible with JPEG will be preserved.

The modification will produce all sizes, regardless whether they are
smaller or bigger than the source (in that case the source size will
be used). This is a change in behaviour I'm currently discussing with
Dan before commiting. For example, the default "High print
resolution" (99999x99999 px) preview size will then produce an
original size JPEG from any image source. Later on, I hope to add a
switch for each size that controls this behaviour and lets you select
to "force" (or not) rendering of a certain size.

Best regards,
Stefan

[ronhof]

Any progress, I am willing to help out debugging/testing/coding/
releasepartying if needed. :-)

-Ron

[Stefan Wild]

Just commited the changes. Please test and let me know!

For now it only shows the preview sizes for TIFF, PSD and EPS. Another
commit today will put the extension selection into config.default.php
and also enable all major Camera RAW formats (if I get the ImageMagick
dcraw delegates to work on my machine) for resizing.

Stefan

[Stefan Wild]

Please note that the camera RAW formats will need the "dcraw" tool to
work. In my case a change in ImageMagick's delegates.xml was needed.
You can test things on the command-line after installing dcraw by
typing "identify nef:DSC_0001.NEF" if your RAW file comes from a Nikon
and is called DSC_0001.NEF. You will need to prepend the file's
extension in lower case and a colon before the filename (e.g.
"cr2:filename.cr2", "dcr:filename.DCR", etc.).

Happy testing...

[lmolmud]

Does this new functionality apply to any PSD or TIFF file that has
been ingested, or only to those ingested post-update? Thanks.

Larry

[Stefan Wild]

The update changes the image processing. So it will apply to those
files ingested post-update, only. If you have not done so already,
please update to the latest revision (368) from last night, which
prevents the duplicate JPEGs (of the same size) from being generated
and greatly improves the Camera RAW file handling.

However, I guess I will be hacking together a little command-line PHP
script to re-generate all preview sizes of all resources some time,
soon.

Stefan

[Tom Gleason]

won't update_previews.php still work?

[ronhof]

Stefan, kudos for this addition. I will be away on a trip for the next
week or so and will be testing it extensively on that trip. I'll
report back any quirks if any is detected. I just wanted to chime in
that I think this is a big step in the right direction for RS.

RS is now starting to become a very competent platform for efficient
media work flow at least for my own usage scenarios.

Funny how thing seem to fall in place, right....

Now I have to tap into how to accommodate/the different needs for
applying permissions to the different resolutions being made.. Could
you shed some light on that subject? Roadmap, ideas, thoughts.. I have
barely started to get used to how this could be utilized and I like
what I see..

-Ron

[Stefan Wild]

@Tom:
Errrr, well, thanks for the hint! 8)

@Ron:
Thanks for testing. I am in contact with Dan to figure out what'll
happen next.You'll know when something comes out of that... =)

Stefan

[lmolmud]

I posted this under a new and separate thread, but am duplicating here
as I suspect the problem may be related to this addition:

I gave myself a clean, parallel install of Rev 453 (after working
with
Rev 348 for a while.)

453 is great; I installed it to get the added functionality of
offering alternative download renditions for TIFF, PSD, etc., and
that
works just fine.

However, I just noticed a problem with the way that this revision
handles – or, rather, does not handle – "generic" thumbs for
unsupported file formats (file types that do not have a meaningful
preview, or cannot be handled by FFMpeg, etc.)

For example, in previous versions, when ingesting an MP3 or WAV file,
RS would give it a generic "Sound"-type thumbnail. Or, when uploading
a SWF file, it would get the generic "Video" thumbnail.
In 453, these files get the usual

16col.jpg
16pre.jpg
16thm.jpg

in the Filestore directory.

However, these show up in the UI as itsy-bitsy teeny-weeny hardly-
there thumbnails that you can barely click on.
I also notice that in previous versions of RS, when a WAV file is
stored, there are no col, pre, or thm JPEGs in the filestore
subdirectory.

I notice that in the "gfx" directory, where the "generic" "typeX"
icons are stored, they are all in GIF format.

Did something get whacked when the new download options were added?
Can anyone else replicate this? Thanks.

– Larry

[Stefan Wild]

Hi Larry,

when I select "Resource Type: Audio" when uploading the Audio
thumbnail appears just fine. While I won't rule out a bug in the
changes you mention, in my opinion this whole part of code is only
reached when all other methods (InDesign, ffmpeg, PDF, etc.) fail.
Please double-check if this has nothing to do with the Resource Type
you select.

Stefan

[lmolmud]

Stefan –

Thanks. I noticed this situation when I started to add resource types
other than "Photo." First, I tried ingesting a WAV file as an "Audio"-
type resource. Got the malformed thumbnail. (It looks like it's only 1
pixel by 1 pixel; this makes it almost impossible to click on in small
or large thumbnail view.

The same thing happened when I tried to ingest a SWF file as a "Video"-
type resource. I expected the generic Video thumbnail, but got the
same small 1x1 pix thumb.

Looking at filestore, RS seems to be generating thm and pre jpegs for
these kinds of files: I believe this is not the correct behavior. In
previous versions, only the resource file would be stored, forcing RS
to look to the generic images in "gfx."

So, I still think something went wrong here. Thanks again for any
further thought on this.

– Larry