JPhototagger, Imatch 3 by Photools and Linux

[chairraver]

Hi all,

I've found JPhototagger through most recent issue of c't Digitale Fotografie and I must say I'm really impressed. And usually a "J" as a first letter indicates a Java program and indeed JPhotoTager is written in Java.

Anyway, here are some introductory remarks to explain my interest in JPhotoTagger. I'm a full time Linux user, however when I started to scan my negatives I did this on a Windows machine (around 2004-2005), since there was no driver for this scanner available for Linux and additionally, there was no decent photo management software available for Linux (digikam was really in its infancy). Therefore I did the scanning under Windows and used photools.com Imatch as the cataloging program and kept the data on the Linux box via a network drive. In the meantime I've completed the scanning process, however tagging is completed for about 75% I think. I've moved the Windows stuff to a VirtualBox Windows virtual machine running on my Linux box since then. While it basically works ok, it's still a hassle constantly switching platforms. I would prefer, if I could find a more Linux natively solution to move the Imatch stuff to Linux.

Fast forward to today, there is now Aftershot Pro from Corel (formerly Bibble5 by Bibblelabs.com, which I own), digikam has matured a lot and there is JPhototagger. And all these programs understand XMP sidecar files and Imatch . However, I quickly learned, that although all read XMP sidecar files doesn't mean, that I get my tags from Imatch into any of the other programs. I can run VBS scripts inside Imatch, so that I can atleast transfer the hierachical keywords from Imatch into the Photoshop namespace of the XMP specification. And with some additional exiftools fiddleing I even managed to transfer some experimental data into one of the other programs, but it's all from perfect.

I'm still trying to figuring out, to which one of the programs I should move to. JPhotoTagger seems to me like another very good alternative solution.

Anyway, to get to the point this weekend I tried to open a photo collection managed by Imatch in a VM though a network drive. I've located two problems, which stopped JPhotoTagger from reading Imatch generated XMP files.

First, the XMP files written by Imatch ended up with uppercase XMP in the Linux file system. JPhotoTagger wasn't prepared for this. And JPhotoTagger would though Execptions because Imatch would write some Unicode-16 or something before the actual XMP XML data. The Imatch file would start like this

0000000 357 273 277   <   x   :   x   m   p   m   e   t   a       x   m
0000020   l   n   s   :   x   =   "   a   d   o   b   e   :   n   s   :

So, with the attached patches I was able to read the Imatch generated XMP files into JPhotoTagger.
--
Cheers Frank

[Elmar Baumann]

Hello Franx,

thanks for the diff. I integrated it with a few changes:
https://code.google.com/p/jphototagger/source/detail?r=98037409b531

Can I have a XMP file created by Imatch for a test? I think, for you
it's no problem to create the JARs with the applied fixes (ant or
NetBeans)? Otherwise or if other users having the same problem, I will
create a new release.

BTW: Did you try VueScan?

Regards,
Elmar
--
http://www.elmar-baumann.de/fotografie/

[chairraver]

Am Montag, 16. Juli 2012 20:15:59 UTC+2 schrieb Elmar:

Hello Franx,

thanks for the diff. I integrated it with a few changes:
https://code.google.com/p/jphototagger/source/detail?r=98037409b531

Can I have a XMP file created by Imatch for a test? I think, for you
it's no problem to create the JARs with the applied fixes (ant or
NetBeans)? Otherwise or if other users having the same problem, I will
create a new release.

No problem, I'm attaching two Imatch generated XMP files. I updated the the mercurial repository.
And now JPhotoTagger works just fine with Imatch XMP files. Thanks a lot.
 

BTW: Did you try VueScan?

Yes, I did check VueScan (sometime in the beginning of my scanning project, I guess more than 5 years ago) however I didn't spend enough time with it. At the time VueScan didn't show me any significant advantage compared to the Konica Minolta Filmscanner DiMAGE Scan Dual IV scanner software, that it would justify the additional cost.
 

Regards,
Elmar
--
http://www.elmar-baumann.de/fotografie/

 
I'm wondering, which source files would I need to be studying if I wanted to implement read support for Digikams XMP tags (possibly as a plugin) for hierachical keyword support, which looks like this:

<rdf:Description rdf:about=""
            xmlns:digiKam="http://www.digikam.org/ns/1.0/">
         <digiKam:TagsList>
            <rdf:Seq>
               <rdf:li>Jahre/2000</rdf:li>
               <rdf:li>Freunde</rdf:li>
               <rdf:li>Ereignisse/Geburtstage/Jara/Feier</rdf:li>
            </rdf:Seq>
         </digiKam:TagsList>
      </rdf:Description>

I don't know if I'm biting off more than I can chew here, but I would atleast like to look at it. My Java skills are a bit dated having not done any serious work for quite some time. I'm basically only a hobbyist hacking a bit with Java on level 1.4 or 1.5. I haven't really mastered 1.6 with generics and enums, etc. yet.
--
Regards,
Frank

[Elmar Baumann]

Hello,

the correct way is modifying the XMP sidecar file and use an existing
helper thread class for updating the database with the new contents of
the XMP file. The XMP should be modified with the provided Adobe XMP
SDK. As long as hierarchical keywords are not standardized (best
candidate is http://www.metadataworkinggroup.org/), JPhotoTagger will
not support them (and if, it's a question of available time). So
Digikam's hierarchy should be flattend (separate keywords). The XMP
standard says, that any application writing hierarchical keywords,
should write any keyword and it's parents up to the root as single
Dublin Core Subjects (keywords) - this is overhead and error prone due
synchronization in both directions. It seems, that Digikam doesn't do
that.

There are two recommended ways to extend JPhotoTagger (combining them
is no problem):

* If selected files should be processed, implement a FileProcessorPlugin,
a simple example is the project
"JPhotoTagger: Plugin: Copy Filenames To Clipboard"
* If a menu entry e.g. under "Tools" should be provided and a bunch of
directories should be processed, implement a module, as example
see "JPhotoTagger: Module: Repository File Browser"

Such work should be done in a separate thread.

If you can wait until if I have time, I can write an interface for
that approach - read from other proprietary sidecar file branches and
merge them with standard (JPhotoTagger) tags, so that all you have to
do is the reading part, the setters will be regular Java methods. The
implementation provides either a context menu entry for selected files
or a complete scan of all known files. First draft:

public interface XmpModifier {
void modifyXmp(File sidecarFile, Xmp xmp); // is
org.jphototagger.domain.metadata.xmp.Xmp
}

An implementation is "plugged in" via Annotations:

@ServiceProvider(service=XmpModifier.class)
public DigikamXmpModifier implements XmpModifier {
@Override
public void modifyXmp(File sidecarFile, Xmp xmp) {
xmp.addToCollection(XmpDcSubjectsSubjectMetaDataValue.INSTANCE,
readKeywords(sidecarFile));
// ...
}
}

Elsewhere e.g. in a module:

@ServiceProvider(service=XmpModifier.class)
public ImatchXmpModifier implements XmpModifier {
@Override
public void modifyXmp(File sidecarFile, Xmp xmp) {
xmp.setValue(XmpDcDescriptionMetaDataValue.INSTANCE,
readDescription(sidecarFile));
// ...
}
}

Regards,
Elmar

--
http://www.elmar-baumann.de/

[Elmar Baumann]

Now the interface is integrated as
org.jphototagger.domain.metadata.xmp.XmpModifier
(https://code.google.com/p/jphototagger/source/detail?r=d93a4f9d30a3),
was not much work. Every provided modifier will be invoked if:

* If a thumbnail is displayed and it's XMP sidecar file's modification
date is newer than it's last database record
* Context menu > Refresh > Update metadata
* File menu > Scan Folders
* File menu > Database > Update: Re-read XMP of all known files

If you wish to read XMP with the included Adobe XMP SDK, I can add a
module skeleton, e.g. DigikamXmpModifier.

[chairraver]

Hi again.

So, in the last couple of days I've tried to make sense of IMatch various Visual Basic scripts, which are included in the distribution. Unfortunately Imatch doesn't allow the direct manipulation of XMP information through the embedded Visual Basic interpreter, but only indirectly through the setting of IPTC information, which then is written to the appropriate XMP tags.

So, in the end I was able to transfer the hierachical keywords from Imatch into IPTC dc:subject XML tags, which JPhototagger would correctly read. The keyword would look something like "Family.Children.<Childs Name>" or what ever. Imatch uses the same dot notation as for instance Java's import statement for class hierachis.

I've come to the conclusion, that I'm going to import all photos with this dot notation into JPhotoTagger. After that I will select all photos with the same keywords, retag them manually with the appropriate single word tags. Then I'm going to delete the dot notation keywords. Overall I had scanned roughly 125 rolls of films. Assuming 36 exposures, that would make roughly 5000 pictures.
I think, that would be the cleanest way to transfer the dot notation into a single word tag notation. Since Imatch also duplicates the transfered hierachical keyword information into the photoshop namespace, I could even keep this info around and use it sometime later.

Am I makeing sense?

I've notice one problem with the uppercase file name extension handling however during my experiments. JPhotoTagger correctly reads the XMP file with an uppercase .XMP, but when I've edited something within JPhotoTagger, the new ly written XMP sidecar file is written with lowercase ".xmp" extension. And the new file doesn't contain all informaton from the original .XMP file. So I've then basically two XMP files for a photo, one with .xmp and one with .XMP.

--
Regards,
Frank

[Elmar Baumann]

On 26.07.2012 20:50, chairraver wrote:
> Am I makeing sense?

The new interface can be used to automate this - not tested!, but it should work:

@ServiceProvider(service=XmpModifier.class)
public IMatchXmpModifier implements XmpModifier {
@Override
boolean modifyXmp(File sidecarFile, Xmp xmp) {
Object value = getValue(XmpDcSubjectsSubjectMetaDataValue.INSTANCE);
List<String> dcSubjects = new ArrayList<String>();
boolean hasHierarchy = false;
if (value instanceof Collection<?>) {
for (String dcSubject : (Collection<String>) value) {
if (dcSubject.contains(".")) {
dcSubjects.addAll(Arrays.<String>asList(dcSubject.split("\\.")));
hasHierarchy = true;
} else {
dcSubjects.add(dcSubject);
}
}
if (hasHierarchy) {
remove(XmpDcCreatorMetaDataValue.INSTANCE); // Clear Old
setValue(XmpDcCreatorMetaDataValue.INSTANCE, dcSubjects);
}
}
return hasHierarchy;

}
}

> I've notice one problem with the uppercase file name extension handling however during my experiments. JPhotoTagger correctly reads the XMP file with an uppercase .XMP, but when I've edited something within JPhotoTagger, the new ly written XMP sidecar file is written with lowercase ".xmp" extension. And the new file doesn't contain all informaton from the original .XMP file. So I've then basically two XMP files for a photo, one with .xmp and one with .XMP.

I will check that, if I have time.

[Elmar Baumann]

On 26.07.2012 20:50, chairraver wrote:

> I've notice one problem with the uppercase file name extension handling however during my experiments. JPhotoTagger correctly reads the XMP file with an uppercase .XMP, but when I've edited something within JPhotoTagger, the new ly written XMP sidecar file is written with lowercase ".xmp" extension. And the new file doesn't contain all informaton from the original .XMP file. So I've then basically two XMP files for a photo, one with .xmp and one with .XMP.

Hello chairraver,

thanks for the bugreport. Should be fixed with revision: 793491b6ff8e:
https://code.google.com/p/jphototagger/source/detail?r=793491b6ff8e43029c0be984c9a1677c19681aaf

[chairraver]

Thanks, XMP file handling concerning upper- and lowercase extensions is working fine now.

Finally this is the code I managed to produce from your example (as I mentioned earlier I'm not really upto date with newer Java stuff, annotations, generics, enums, etc) it took a little while.

package org.jphototagger.domain.metadata.xmp;

import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import org.openide.util.lookup.ServiceProvider;

@ServiceProvider(service=XmpModifier.class)
public final class ImatchXmpModifier implements XmpModifier {

    @Override
    public boolean modifyXmp(File sidecarFile, Xmp xmp) {
        Object value = xmp.getValue(XmpDcSubjectsSubjectMetaDataValue.INSTANCE);

        List<String> dcSubjects = new ArrayList<String>();
        boolean hasHierarchy = false;
        if (value instanceof Collection<?>) {
            for (String dcSubject : (Collection<String>) value) {
                if (dcSubject.contains(".")) {
                    dcSubjects.addAll(Arrays.<String>asList(dcSubject.split("\\.")));
                    hasHierarchy = true;
                } else {
                    dcSubjects.add(dcSubject);
                }
            }
            if (hasHierarchy) {

                xmp.remove(XmpDcSubjectsSubjectMetaDataValue.INSTANCE); // Clear Old
                xmp.setValue(XmpDcSubjectsSubjectMetaDataValue.INSTANCE, dcSubjects);
            }
        }
        return hasHierarchy;
    }
}

Am I correct to assume, that the XMP code takes care of multiple, identical values? For instance the two hierachical expressions

Family.Children.Jane
Events.Birthday.Party.Jane

would produce two dc:subjects once split with value Jane.

Anyway, I'm quite a bit more comfortable to use Java to automatically fine tune the XMP data instead of the VBS that Imatch uses.

Again, thank you for your support.
--
Cheers
Frank

[Elmar Baumann]

On 29.07.2012 22:51, chairraver wrote:
> package org.jphototagger.domain.metadata.xmp;

If a feature is requested often, it will be integrated into JPhotoTagger
(if I have time or onother - not yet existing - developer will do it).
For other tasks it's better to use the project "User Services" (added
today, before it must be organized manually) which will not have a
default implementation and will never be part of a distribution.
Otherwise you have always rebuild JPhotoTagger by yourself if you want
to be up to date. See http://www.jphototagger.org/dev/user_services.html
- especially the paragraph "User Services' project skeleton".

> Am I correct to assume, that the XMP code takes care of multiple,
> identical values? For instance the two hierachical expressions
>
> Family.Children.Jane
> Events.Birthday.Party.Jane
>
> would produce two dc:subjects once split with value Jane.

Equal strings occur only one times, if you add "Jane" five times as
dublin core subject, it will be added only once (but the XMP object
doesn't take care of white spaces, so " Jane", "Jane" or "Jane " are
three different subjects, to avoid this, trim the string before adding it).

[chairraver]

Am Montag, 30. Juli 2012 20:39:04 UTC+2 schrieb Elmar:

On 29.07.2012 22:51, chairraver wrote:
> package org.jphototagger.domain.metadata.xmp;

If a feature is requested often, it will be integrated into JPhotoTagger
(if I have time or onother - not yet existing - developer will do it).
For other tasks it's better to use the project "User Services" (added
today, before it must be organized manually) which will not have a
default implementation and will never be part of a distribution.
Otherwise you have always rebuild JPhotoTagger by yourself if you want
to be up to date. See http://www.jphototagger.org/dev/user_services.html
- especially the paragraph "User Services' project skeleton".

Oh, I didn't want to introduce a new feature or some such. It was simply, that I had Netbeans open at that position and I pasted the code into that window.

I'm going to keep the  "User Services' project skeleton" in mind.

Thanks.
--
Frank