(probably :-) GREP-question regarding Photos-metadata

106 views
Skip to first unread message

Vlad Ghitulescu

unread,
Mar 13, 2020, 8:37:02 AM3/13/20
to BBEdit Talk

Hey!

This is a BBEdit-question, but I first need to give a little background to it. :-)

As all photographers here already know, the metadata to the photos lives in a suplimentary XMP-file.
Here is a sample of a XMP-file for one of my photos, where I filled in the so called IPTC-keywords in Photo Mechanic:

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.6.0">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description rdf:about=""
        xmlns:dc="http://purl.org/dc/elements/1.1/"
        xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
        xmlns:xmp="http://ns.adobe.com/xap/1.0/"
        xmlns:photomechanic="http://ns.camerabits.com/photomechanic/1.0/"
     photoshop:DateCreated="2018-05-10T13:35:00.03"
     xmp:CreateDate="2018-05-10T13:35:00.03"
     xmp:Rating="0"
     photomechanic:ColorClass="0"
     photomechanic:Tagged="False"
     photomechanic:Prefs="0:0:0:-00001"
     photomechanic:PMVersion="PM6">
     <dc:subject>
        <rdf:Bag>
         <rdf:li>Ben</rdf:li>
         <rdf:li>Mina</rdf:li>
         <rdf:li>Tom</rdf:li>
         <rdf:li>Vlad</rdf:li>
        </rdf:Bag>
     </dc:subject>
    </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

The individual data is the creation-date:

     photoshop:DateCreated="2018-05-10T13:35:00.03"
     xmp:CreateDate="2018-05-10T13:35:00.03"

and the IPTC-keywords:

         <rdf:li>Ben</rdf:li>
         <rdf:li>Mina</rdf:li>
         <rdf:li>Tom</rdf:li>
         <rdf:li>Vlad</rdf:li>

Another background-information that probably all of you know: there is no relation between the IPTC / XMP - keywords and the Finder / macOS - tags.

And this exactly is my problem.

I need to input somehow / somewhere the metadata to my photos, but I don't want to make this twice in order to have this information present in the IPTC-keywords AND in the Finder-tags as well.

This is not new at all: Google-ing about "copy IPTC-keywords into Finder-tags" or viceversa I've found a lot but all of it involved in one step or another the EXIFtool, a command line utility that reads and writes photo-metadata… and this doesn't scale very well (I have A LOT of photos: about 220,000 pics that are more than 2 TB big!) and it is slow, because the EXIFtool have to read the metadata from the photo.

So my idea is to take care or the IPTC-keywords in Photo Mechanic (this would create a XMP-file as the one above for every photo I have) and then - and here comes BBEdit! :-) - somehow transfer the data from the XMP-file (that is in the end only a text-file!) to the Finder-tags.

Now finally to my question (sorry for the length!): How could I

  • take each IPTC-keyword (delimited by the tag < rdf:li >, see above) and
  • write this IPTC-keyword into a Finder-tag

for every photo in a folder?

Ideally I'd have a script, point it to a folder full of photos and BOOM!, after this every photo would have the same Finder-tags as the IPTC-keywords!

Could you please help me with this?

Thanks in advance!

Regards,
Vlad

Vlad Ghitulescu

unread,
Mar 13, 2020, 8:45:29 AM3/13/20
to BBEdit Talk

I was inexact regarding file-names and -number:

  • the XMP-file and the photo-file have the same name AND
  • there is most of the time more than one photo-file, because I shoot in RAW (that’s CR2 for me, shooting Canon) and all of the edited versions are JPG, once in a while TIFF.

So for a given photo „PhotoName“ I have always at least two files

PhotoName.CR2 (the RAW-file)
PhotoName.XMP (the metadata-file, I was hard-working enough to take care of the metadata ;-)

and for the most of the photos there are even some other edited versions as

PhotoName.JPG
PhotoName-1.JPG
PhotoName-bw.JPG
PhotoName.TIFF

The edited versions all begin with A but can have a) some data added to the end of the file-name (as digits for various versions or -bw for black & white) and b) some other file-extensions (JPG and once in while TIFF).

That make my dream-script a little more complicated:

  1. take this photo-folder;
  2. read from the first XMP-file the IPTC-keywords;
  3. copy all of this IPTC-keywords as Finder-tags to all of the files with the suffixes CR2, JPG or TIFF of which the file-name begins with the same name as the XMP-file;
  4. loop within the photo-folder until there is no XMP-file anymore.

I hope that now is more clearer.

Thanks in advance!

Regards,
Vlad

--
This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: https://twitter.com/bbedit
--- You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbedit/93FE2214-4BDD-4C41-952F-61C910E44C38%40Ghitulescu.de.

Maarten Sneep

unread,
Mar 13, 2020, 12:06:30 PM3/13/20
to bbe...@googlegroups.com
Hi,

If you need to work with xml I recommend to use a real xml parser. I prefer to use python with lxml. It will allow you to exploit the hierarchy of the xml file in ways that grep will not allow you to do.

Best,

Maarten

Vlad Ghitulescu

unread,
Mar 13, 2020, 12:25:15 PM3/13/20
to bbe...@googlegroups.com

Maarten,

thanks for your quick reply!

I mentioned grep because I'd like to search for the keywords that are recognizable due to the rfd:li - tags in the XMP-file AND also because this is something I suppose it's easy to do with grep in BBEdit (even if it's still out of my reach :-)… as well as the rest of my wish-script :-/).

Thanks again!

Regards,
Vlad

--
This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: <https://twitter.com/bbedit>
---
You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.

Vlad Ghitulescu

unread,
Mar 14, 2020, 11:11:05 AM3/14/20
to BBEdit Talk

Hey!

What I've got so far:

  1. I manage to extract the IPTC-keywords from the initial example:

using BBEdit's "Process Line Containing…"-command, where I searched for

     <rdf:li>
  1. I replaced (manually, not via a script etc.) then

  • first

     <rdf:li>

  • and then

    </rdf:li>

with nothing.

The result:

Ben
Mina
Tom
Vlad

are the 4 IPTC-Keywords that I want to set as Finder-tags to the CR2 / JPG / TIFF - file with the same name as the XMP-file.

  1. I've found an Automator-action that can assign tags to files or folders.

These are only a couple of pieces of the whole puzzle and right now I don't have a clue how to "glue" them together :-(

Any help would be greatly appreciated.

Regards,
Vlad

Bildschirmfoto 2020-03-14 um 15.39.50.png

Jean-Christophe Helary

unread,
Mar 14, 2020, 11:20:04 AM3/14/20
to bbe...@googlegroups.com
Vlad,

You're just a few AppleScript lines away from solving your project.

The problem is that Finder's dictionary does *not* provide access to "modern" tags. So we have to code that with AS but we don't have to reinvent the wheel since Shane Stanley has already written *the* code that you can use to manipulate tags in AS.

I am not sure the link points at the canonical version, but that's the one I use for reference:

https://lists.apple.com/archives/applescript-users/2015/Jan/msg00193.html

Jean-Christophe

> On Mar 14, 2020, at 23:44, Vlad Ghitulescu <Vl...@Ghitulescu.de> wrote:
>
> Hey!
>
> What I've got so far:
>
> • I manage to extract the IPTC-keywords from the initial example:
> • I replaced (manually, not via a script etc.) then
> • first
>
> <rdf:li>
>
> • and then
>
> </rdf:li>
>
> with nothing.
>
> The result:
>
> Ben
> Mina
> Tom
> Vlad
>
> are the 4 IPTC-Keywords that I want to set as Finder-tags to the CR2 / JPG / TIFF - file with the same name as the XMP-file.
>
> • I've found an Automator-action that can assign tags to files or folders.
> <Bildschirmfoto 2020-03-14 um 15.39.50.png>
> The individual data is the creation-date:
>
> photoshop:DateCreated="2018-05-10T13:35:00.03"
> xmp:CreateDate="2018-05-10T13:35:00.03"
>
> and the IPTC-keywords:
>
> <rdf:li>Ben</rdf:li>
> <rdf:li>Mina</rdf:li>
> <rdf:li>Tom</rdf:li>
> <rdf:li>Vlad</rdf:li>
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/bbedit/74E567C0-F5BE-48D8-A30F-9EEBEA2737C7%40Ghitulescu.de.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune


Vlad Ghitulescu

unread,
Mar 14, 2020, 12:32:26 PM3/14/20
to bbe...@googlegroups.com

Jean-Christophe,

thanks a lot for your reply!

Google-ing (see my first email) I've found already the very same AppleScript from Shane Stanley back in 2002 (!) - so yes, that's another puzzle piece! Hurray! :-)

I'm still missing a lot:

  1. I don't know how to peek the first XMP-file in the chosen folder and then apply the magic of extracting the IPTC-keywords of it. This is what I've manually done with BBEdit, but I don’t know how to put it in an AppleScript / RegEx :-(

  2. How do I pipe the result from point 1 to Shane Stanley's AppleScript? (*)

  3. And how do I tell to the above AppleScript to which CR2 / JPG / TIFF - file (with the same name as the XMP-file from point 1 above) to apply the tags (= IPTC-keywords) extracted from the XMP-file in point 1 above?!

  4. And finally: How do I loop to the next XMP-file within the chosen folder?

It feels like I have less than I'm missing right now! :-D

At least the points 1, 2 and 4 seem trivial… I'll google them.

Thanks again, Jean-Christophe!

Regards,
Vlad

(*): I've found a hint from Dr. Drang in an AppleScript called from KeyboardMaestro to tag a file. What I like about it is that it uses a list of tags… that could be the result of my point 1 above! :-) Ok, I still don't know how to pipe it to Keyboard Maestro, so I'm still on square (= point ;-) 1! :-(

Vlad Ghitulescu

unread,
Mar 14, 2020, 1:28:25 PM3/14/20
to bbe...@googlegroups.com

I've found here on StackOverflow how to:

  • choose a folder
  • filter only the XMP-files
  • store the name of the XMP-files in an array

Here's the modified part of the AppleScript:

-- we can check the file extensions of a file against this list to evaluate if it's a XMP file
set xmp_ext_list to {"xmp"}

-- get the folder to check
set f to choose folder

-- notice the use of "entire contents" to also go through subfolders of f
-- use a "whose" filter to find only the video files
tell application "Finder"
    set xmpFiles to (files of entire contents of f whose name extension is in xmp_ext_list) as alias list
end tell

I'm still searching for the missing pieces… :-)

GP

unread,
Mar 14, 2020, 2:04:00 PM3/14/20
to BBEdit Talk
Do you know "the EXIFtool, a command line utility" is just a wrapper for a perl module you can download the source code  for from CPAN?

If you dig down into the perl source code you'll find the regular expressions used by EXIFtool to parse XMP file data.  Since BBEdit uses the PCRE library for grep searching, whatever regular expressions you find in perl code can work with BBEdit (albeit you may have to do some work if there are precompiled regular expression variables involved).

Also, EXIFtool can read .XMP files. So, if you have a faster means of extracting the metadata from photo files and creating the .XMP files, you can use that faster means and still use EXIFtool code on the trimmed down metadata only .XMP files.

Also, for your dream script, write a perl script which imports the EXIFtool perl module which:

1. For a given input directory, uses the glob function to build a list of files of *.XMP files
2. Iterate over the list of XMP files and use the routine(s) from the EXIFtool module to extract the desired tags
3. Extract the base file name from the XMP file name you've just extracted the tags from and then glob another file list (e.g., base*.JPG, base*.TIFF)
4. Iterate over the file list created in 3 writting out to each file the tags extacted from the specific XMP file iteration

As far as scaling concerns goes, that's a lot of files to be writing metadata out to the file system on so I don't think you're going to find any solution that's very speedy.  As I think you're hinting at, anything that involves a manual step per file isn't a viable solution.

From your follow up postings I see you're trying to find a scripting type solution using an automated (scripted?) combination of BBEdit features and yet to be determined glue.  However great a development tool BBEdit is, I don't think trying for a BBEdit specific solution is a good first choice to pursue.  

On Friday, March 13, 2020 at 5:37:02 AM UTC-7, Vlad Ghitulescu wrote:

Hey!

This is a BBEdit-question, but I first need to give a little background to it. :-)

 

[snip]

 

This is not new at all: Google-ing about "copy IPTC-keywords into Finder-tags" or viceversa I've found a lot but all of it involved in one step or another the EXIFtool, a command line utility that reads and writes photo-metadata… and this doesn't scale very well (I have A LOT of photos: about 220,000 pics that are more than 2 TB big!) and it is slow, because the EXIFtool have to read the metadata from the photo.

 

[snip]

Vlad Ghitulescu

unread,
Mar 14, 2020, 2:04:01 PM3/14/20
to bbe...@googlegroups.com

I run the little piece of AppleScript below supplemented with a loop within the xmpFiles (only to see that I'm able to loop… now at least you see that I don't know exactly what I'm doing! ;-):

-- we can check the file extensions of a file against this list to evaluate if it's a XMP file
set xmp_ext_list to {"xmp"}

-- get the folder to check
set photoFolder to choose folder

-- notice the use of "entire contents" to also go through subfolders of photoFolder
-- use a "whose" filter to find only the XMP files
tell application "Finder"
    set xmpFiles to (files of entire contents of photoFolder whose name extension is in xmp_ext_list) as alias list
end tell

set filePaths to {}
repeat with currentXMPfile in xmpFiles
    set end of filePaths to {POSIX path of currentXMPfile}
end repeat

on a folder containing two subfolders with a total of 237 RAW (= CR2)-files and (only) 25 XMP-files and it took about 15 seconds (!) to finish.

I don't have any empirical values to compare with but: Is it not a little slow for only

  • filter from 237 + 25 = 262 files
  • spread over only two subfolders
  • the 25 XMP-files

?!

Could it be that AppleScript is not the perfect tool at least for this part?!

Regards,
Vlad

Vlad Ghitulescu

unread,
Mar 14, 2020, 3:52:11 PM3/14/20
to BBEdit Talk

GP, thanks for your detailled email.

No, I didn't knew that the EXIFtool parse the XMP file data.
(Now that you said this it seems obvious to me too - I thought initially that the EXIFtool will somehow read the photo-file, but that's nonsense, since the IPTC-data is in the XMP-file ant not in the RAW / JPG / TIFF - file itself).

Your 4-steps "formula" sounds much more valid than my dream script.
I understand on the high level what you're describing here but unfortunately I have no clue how to implement it since I haven't had the chance to dig into perl yet.

Let me clarify the only manual step that I still see in the whole process: I'll have to set the IPTC-metadata (at least the basic one like caption, keywords, author, copyright etc.) for each and every photo-file (CR2, JPG, TIFF).

Only after this "manual labor" :-) I'd like to choose either the root-folder of all my photo-subfolders or - if this will be too slow - each photo-subfolder and let the script automatically copy the IPTC-keywords of each photo-file (from the XMP-file, via EXIFtool) into its Finder-tags.

As I said initially, I thought about BBEdit first because BBEdit Talk is filled with GREP / regex - questions and it seem like a piece of cake (iterate over a series of text-files, parse the IPTC-keywords and pipe them into something that tag the photo-file with it… easy, right?! ;-)

If my question is actually inappropriate here or better suited elsewhere I'm sorry and totally grateful for each and every hint, because: I still have this HUGE PILE of photos to tag! ;-) :-(

Thanks again, GP!

Regards,
Vlad

--
This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: <https://twitter.com/bbedit>
---
You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.
Message has been deleted

Scott in Pollock

unread,
Mar 14, 2020, 10:27:34 PM3/14/20
to BBEdit Talk
Vlad,

While I am sure this is doable in AppleScript and or Automator, my initial tests here show it will be slow as all get out (filtering folder contents on file extension, and setting finder labels, reiteratively).

Why not just get a proper cataloging tool like NeoFinder: https://www.cdfinder.de/en/downloads.html, that will automatically read the xmp files into its database, create thumbnails from your images, and provide search and smart collections based on all of your metadata both exif and xmp, with full editing ability, and do it super fast. It is what I use and I couldn't be happier with it.

Vlad Ghitulescu

unread,
Mar 15, 2020, 9:48:46 AM3/15/20
to BBEdit Talk

Scott,

Thank you for your reply!

First the very short answers:

  • I did bought a proper cataloging tool (and would probably buy another one soon :-)
  • none of the Digital Assets Managements - tool I tested can copy IPTC-keywords into Finder-tags.

Now with more details: Just last week I finished exporting my masters and their edited copies from Aperture.
Aperture had a "proper cataloging" and it managed exceptionally IPTC- and EXIF-metadata. But Apple abandoned Aperture many years ago (and Aperture cannot run under Catalina - that's the single reason I'm still on Mojave right now) and here I am with all my painfully completed Aperture-keywords lost!

So I decided to never use a proprietary system for search & find photos again.

Meanwhile the XMP-files are the standard to go, so I tested last week:

  • Photo Mechanic 6
  • Photo Supreme
  • Graphic Converter
  • NeoFinder
  • Lyn
  • Darktable
  • Emulsion

In the end I've got Photo Mechanic (it's by far the fastest when culling and can assign metadata on import) and I'm still testing Neo Finder.

But not a single DAM software I tested is able to copy the IPTC-keywords into Finder-tags - and I asked directly NeoFinder's Norbert Dörner and talked with Photo Mechanic's support about it as well.

This is what I try to solve with a script - in order to be able to filter (within a DAM AND / OR in the Finder or anything else that uses Spotlight, like HoudahSpot, what I very much prefer) my photos by the Finder-tags.

I'm probably safe enough using only IPTC-metadata now but I'm still slightly traumatized ;-) after the Aperture-episode and I would like to have the Finder-tags assigned to my photos as well, if it's possible.

Sorry for the long answer & thanks again!

Regards,
Vlad

On 14 Mar 2020, at 21:29, Scott in Pollock wrote:

Vlad,

While I am sure this is doable in AppleScript and or Automator, my initial tests here show it will be slow as all get out (filtering folder contents on file extension, and setting finder labels, reiteratively).

Why not just get a proper cataloging tool like [url=https://www.cdfinder.de/en/downloads.html]NeoFinder[/url], that will automatically read the xmp files into its database, create thumbnails from your images, and provide search and smart collections based on all of your metadata both exif and xmp, with full editing ability, and do it super fast. It is what I use and I couldn't be happier with it.

--
This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: <https://twitter.com/bbedit>
---
You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.

Jeffrey Jones

unread,
Mar 15, 2020, 9:03:40 PM3/15/20
to bbe...@googlegroups.com
Vlad,

I don't understand what you are trying to accomplish. Why do you want to use Finder Tags when photo files already have a standard tagging system, IPTC, built in? JPEG, TIFF, PNG, HEIF all contain IPTC in the file. (I don't know about Canon, but Nikon raw files also contain IPTC.) There is no reason at any point to resort to sidecar files -- the IPTC is in the photo files themselves.

The IPTC data is accessible in Spotlight. For photos, I don't see any advantage for using Finder Tags instead of IPTC. IPTC data is carried across different software, file formats, computer platforms. Finder Tags work only in the Finder.

I migrated from Aperture to Capture One Pro. (It is a little more expensive than some of the other DAM options, but they have a 30-day free trial.) I chose it because it did a superb job of importing my Aperture library. All metadata is carried over, as well as organization -- projects, albums, labels, ratings. Even some (but not all) adjustments.

Scott in Pollock

unread,
Mar 15, 2020, 9:03:52 PM3/15/20
to BBEdit Talk
It may be possible, but I have yet to get it to work with AppleScript in Hi-C Error, despite trying some supposedly working examples on the web. This is not surprising as the examples are pretty old (from back when tags started in Mavericks), and Apple has a history of breaking AppleScript stuff and never getting around to fixing it.

You should be able to use xattr from the command line, but it requires the tag strings to be input as a pList. So one must first query the current tags, massage that data into XML, and then add the tags to be added, and input that pList back to the xattr command.

There appears to be a command line tool named 'tag' that simplifies this process greatly, but I have yet to find a compiled binary of it. You either need to have MacPorts or Homebrew set up to install it, or have Xcode installed so you can build it yourself (none of which I have here ATM).

Vlad Ghitulescu

unread,
Mar 16, 2020, 7:34:32 AM3/16/20
to BBEdit Talk

Hi, Scott!


Yes, it looks like I'm out of luck! ;-)

Thanks anyway!


Regards,
Vlad

--
This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: <https://twitter.com/bbedit>
---
You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.

Vlad Ghitulescu

unread,
Mar 16, 2020, 7:34:33 AM3/16/20
to 'Jeffrey Jones' via BBEdit Talk

Hi, Jeffrey!

Thanks for your reply!

I’m aware that this reply is entirely NOT related to BBEdit so I’ll keep it short - let’s talk private about this.

On 15 Mar 2020, at 20:38, 'Jeffrey Jones' via BBEdit Talk wrote:

The IPTC data is accessible in Spotlight.

I couldn't get to any IPTC-metadata through Spotlight:

No XMP-metadata:

Only 2 EXIF-attributes:

The above screenshots are from a Finder-search-window, but I get the same within HoudahSpot that uses the Spotlight-index as well.

In Automator I have only one action regarding IPTC (set IPTC-attributes) from Aperture (still on my system).

Regards,
Vlad

Bildschirmfoto 2020-03-16 um 07.43.10.png
Bildschirmfoto 2020-03-16 um 07.43.26.png
Bildschirmfoto 2020-03-16 um 07.43.37.png

Jan Erik Moström

unread,
Mar 16, 2020, 8:57:23 AM3/16/20
to 'Jeffrey Jones' via BBEdit Talk
On 16 Mar 2020, at 7:50, Vlad Ghitulescu wrote:

> I couldn't get to any IPTC-metadata through Spotlight:

This is something where you should use a tool like https://exiftool.org

= jem

Jan Erik Moström

unread,
Mar 16, 2020, 9:00:45 AM3/16/20
to 'Jeffrey Jones' via BBEdit Talk
And if you want some kind of DAM tool then I would suggest looking at
for example

- Lightroom
- Capture One
- DxO PhotoLab
- Lyn
- OnOne (don't know anything about this)
- Darktable (don't know anything about this)

It all depends what kind of requirements you have.

= jem
Reply all
Reply to author
Forward
0 new messages