How can I harvest images/movies from Messages?

[Tim Lance]

I always knew I'd rue the days I didn't merely right-click practically every
image/movie of my grandkids (and a few other folks) and choose 'Add to Photos
Library'.

Is there any easy way to go back and harvest them all? We're talking texts
going back to the original iPhone days! If there isn't an app there damn well
should be. Perhaps it's simply scriptable?

I know the process would take some serious time as when I go back not too far
the item must be redownloaded.

Thanks for any consideration.

[Lewis]

In message <Mts3H.197080$r25....@fx08.iad> Tim Lance <n...@here.org> wrote:
> I always knew I'd rue the days I didn't merely right-click practically every
> image/movie of my grandkids (and a few other folks) and choose 'Add to Photos
> Library'.

> Is there any easy way to go back and harvest them all? We're talking texts
> going back to the original iPhone days! If there isn't an app there damn well
> should be. Perhaps it's simply scriptable?

If you set iChat /iMessage/Messages to always keep backups then yes,you
have all those old posts If you didn't, you don't.

> I know the process would take some serious time as when I go back not too far
> the item must be redownloaded.

Not really. I created an archive of all the png/jpg images in my Message
hsitory in a couple of minutes using the command line.


--
"It was a dark and stormy night; the rain fell in torrents -- except
at occasional intervals, when it was checked by a violent gust of
wind which swept up the streets (for it is in London that our
scene lies) rattling along the housetops, and fiercely agitating
the scanty flame of the lamps that struggled against the
darkness."

[Lewis]

In message <hr7cdd...@mid.individual.net> Jolly Roger <jolly...@pobox.com> wrote:

> On 2020-09-01, Lewis <g.k...@gmail.com.dontsendmecopies> wrote:
>>
>> Not really. I created an archive of all the png/jpg images in my Message
>> hsitory in a couple of minutes using the command line.

> Nice. Share details, please. : )

cd ~/Library/Messages/Attachments
cp **/*.jpg /target/folder
cp **/*.png /target/folder

(I could have combined the copies into one line, but I didn’t)

--
"Are you pondering what I'm pondering?"
"I think so, Brain, but if I have my portrait drawn, will we have
time to make it to the lifeboats?"

[Tim Lance]

On Sep 1, 2020, Lewis wrote
(in article <slrnrkt38l....@ProMini.lan>):

> In message<hr7cdd...@mid.individual.net> Jolly Roger
> <jolly...@pobox.com> wrote:
> > On 2020-09-01, Lewis<g.k...@gmail.com.dontsendmecopies> wrote:
> > >
> > > Not really. I created an archive of all the png/jpg images in my Message
> > > hsitory in a couple of minutes using the command line.
>
> > Nice. Share details, please. : )
>
> cd ~/Library/Messages/Attachments
> cp **/*.jpg /target/folder
> cp **/*.png /target/folder
>
> (I could have combined the copies into one line, but I didn’t)

So much thanks. However my skill with UNIX extends to my ability to
copy/paste. I hope you are willing to continue your help.

Below I paste what I got. I have no clue what to do.

mes-MacBook-Pro:~ me$ cd ~/Library/Messages/Attachments

mes-MacBook-Pro:Attachments me$ cp **/*.jpg /~/Pictures/Messages

cp: **/*.jpg: No such file or directory

mes-MacBook-Pro:Attachments me$ cp **/*.jpg/~/Pictures/Messages

usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file target_file

cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file ... target_directory

mes-MacBook-Pro:Attachments me$

—

t

[Tim Lance]

On Sep 1, 2020, Tim Lance wrote
(in article <0ky3H.296313$0W4....@fx42.iad>):

Is it something about the fact that there are three more weirdly named folder
levels between ~/Library Messages/Attachments and actual files?

For example the first image to be found is in
~/Library Messages/Attachments/0a/10/BFB7C50F-C7BB-4FD2-A994-0268EFE88AF7
and the next is in
~/Library Messages/Attachments/0c/12/0AC8A8E0-277B-4A08-AD30-B1E189FFBDCD

—

t

[David Ritz]

On Tuesday, 01 September 2020 15:39 -0500,
in article <0ky3H.296313$0W4....@fx42.iad>,

Tim Lance <n...@here.org> wrote:

> On Sep 1, 2020, Lewis wrote
> (in article <slrnrkt38l....@ProMini.lan>):

>> cd ~/Library/Messages/Attachments
>> cp **/*.jpg /target/folder
>> cp **/*.png /target/folder

>> (I could have combined the copies into one line, but I didn’t)

> So much thanks. However my skill with UNIX extends to my ability to
> copy/paste. I hope you are willing to continue your help.

Lewis seems to be missing some slashes ("/") in his commands, which
indicate directories.

cd ~/Library/Messages/Attachments
cp */*/*.jpg [targetfolder]
cp */*/*.png [targetfolder]

It is up to you to create the target folder, where "[targetfolder]" is
the full path to the directory where you want the copies saved.

I have files with both upper case and lower case file extensions, in
~/Library/Messages/Attachments/. You may wish to include:

cp */*/*.JPG [targetfolder]

--
David Ritz <dr...@mindspring.com>
Be kind to animals; kiss a shark.

[Tim Lance]

On Sep 1, 2020, David Ritz wrote
(in article<alpine.OSX.2.21.2...@mako.ath.cx>):

> On Tuesday, 01 September 2020 15:39 -0500,
> in article <0ky3H.296313$0W4....@fx42.iad>,
> Tim Lance <n...@here.org> wrote:
>
> > On Sep 1, 2020, Lewis wrote
> > (in article<slrnrkt38l....@ProMini.lan>):
>
> > > cd ~/Library/Messages/Attachments
> > > cp **/*.jpg /target/folder
> > > cp **/*.png /target/folder
>
> > > (I could have combined the copies into one line, but I didn’t)
>
> > So much thanks. However my skill with UNIX extends to my ability to
> > copy/paste. I hope you are willing to continue your help.
>
> Lewis seems to be missing some slashes ("/") in his commands, which
> indicate directories.
>
> cd ~/Library/Messages/Attachments
> cp */*/*.jpg [targetfolder]
> cp */*/*.png [targetfolder]
>
> It is up to you to create the target folder, where "[targetfolder]" is
> the full path to the directory where you want the copies saved.
>
> I have files with both upper case and lower case file extensions, in

> > /Library/Messages/Attachments/. You may wish to include:
>
> cp */*/*.JPG [targetfolder]

Give Lewis & David a coupla boxes of cigars!

cp **/*/*/*.jpg ~/Pictures/Messages
cp **/*/*/*.jpeg ~/Pictures/Messages

and so on did the trick.

My whole extended family thanks you.

—

t

[Lewis]

In message <0ky3H.296313$0W4....@fx42.iad> Tim Lance <n...@here.org> wrote:
> On Sep 1, 2020, Lewis wrote
> (in article <slrnrkt38l....@ProMini.lan>):

>> In message<hr7cdd...@mid.individual.net> Jolly Roger
>> <jolly...@pobox.com> wrote:
>> > On 2020-09-01, Lewis<g.k...@gmail.com.dontsendmecopies> wrote:
>> > >
>> > > Not really. I created an archive of all the png/jpg images in my Message
>> > > hsitory in a couple of minutes using the command line.
>>
>> > Nice. Share details, please. : )
>>
>> cd ~/Library/Messages/Attachments
>> cp **/*.jpg /target/folder
>> cp **/*.png /target/folder
>>
>> (I could have combined the copies into one line, but I didn’t)

> So much thanks. However my skill with UNIX extends to my ability to
> copy/paste. I hope you are willing to continue your help.

> Below I paste what I got. I have no clue what to do.

> mes-MacBook-Pro:~ me$ cd ~/Library/Messages/Attachments

> mes-MacBook-Pro:Attachments me$ cp **/*.jpg /~/Pictures/Messages

~/Pictures/Messages without the leading /, but it looks like there are
no jpg files in the Message Attachments folder.

--
SHERRI DOES NOT "GOT BACK" Bart chalkboard Ep. AABF07

[Lewis]

No. ** expands to "all directories under this one" in bash and zsh. If
you changed your shell or you are using a really old OS X version...

--
Whoever had created humanity had left in a major design flaw. It was
its tendency to bend at the knees. --Feet of Clay

[Tim Lance]

On Sep 1, 2020, Lewis wrote

(in article <slrnrku0nm....@ProMini.lan>):

> In message<0ky3H.296313$0W4....@fx42.iad> Tim Lance<n...@here.org> wrote:
> > On Sep 1, 2020, Lewis wrote
> > (in article<slrnrkt38l....@ProMini.lan>):
>
> > > In message<hr7cdd...@mid.individual.net> Jolly Roger
> > > <jolly...@pobox.com> wrote:
> > > > On 2020-09-01, Lewis<g.k...@gmail.com.dontsendmecopies> wrote:
> > > > >
> > > > > Not really. I created an archive of all the png/jpg images in my Message
> > > > > hsitory in a couple of minutes using the command line.
> > >
> > > > Nice. Share details, please. : )
> > >
> > > cd ~/Library/Messages/Attachments
> > > cp **/*.jpg /target/folder
> > > cp **/*.png /target/folder
> > >
> > > (I could have combined the copies into one line, but I didn’t)
>
> > So much thanks. However my skill with UNIX extends to my ability to
> > copy/paste. I hope you are willing to continue your help.
>
> > Below I paste what I got. I have no clue what to do.
>
> > mes-MacBook-Pro:~ me$ cd ~/Library/Messages/Attachments
>
> > mes-MacBook-Pro:Attachments me$ cp **/*.jpg /~/Pictures/Messages
>

> > /Pictures/Messages without the leading /, but it looks like there are
> no jpg files in the Message Attachments folder.

I found 50+ jpg files and almost 600 jpeg. Several more going uppercase on
extensions.

But I just noticed the real problem is they only go back to 2018 when I have
texts with attachments going back many more years. Drilling down in the
~/Library/Messages directory shows nothing prior to 2018.

Back to being bummed..

—

t

[Lewis]

Case does not matter, unless you (or someone else) has made changes on
your Mac's terminal. jpeg would have to be a separate step.*

> But I just noticed the real problem is they only go back to 2018 when I have
> texts with attachments going back many more years. Drilling down in the
> ~/Library/Messages directory shows nothing prior to 2018.

You have texts with attachments WHERE?

* Yes yes, I know, Shush, up.

--
"Are you pondering what I'm pondering?"

"I think so, Brain, but I find scratching just makes it worse."

[Tim Lance]

On Sep 2, 2020, Lewis wrote
(in article <slrnrkurmi....@ProMini.lan>):

> > > /Library/Messages directory shows nothing prior to 2018.
>
> You have texts with attachments WHERE?
>
> * Yes yes, I know, Shush, up.

Texts in the Messages app go back before 2015. Some of those texts have pix
or movies. But the attachments in ~/Library/Messages/Attachments only go back
to 2018.

Thanks for sticking with me.

—

t

[Lewis]

In the messages app ON YOUR MAC?

This makes no sense, if the app is showing the attachments they must be
on your computer.

There are also files in the folders with the suffix
pluginPayloadAttachment but I haven't looked into those. I assumed they
help URLs and previews of those URLs.

Nearly all my message are from other Messages users, and not from SMS
message. This may make a difference?

> Thanks for sticking with me.

--
A closed mouth gathers no feet.

[Lewis]

In message <slrnrkvj44....@ProMini.lan> Lewis <g.k...@gmail.com.dontsendmecopies> wrote:

> There are also files in the folders with the suffix
> pluginPayloadAttachment but I haven't looked into those. I assumed they
> help URLs and previews of those URLs.

Ah, nope, these are a variety of different types of files, all with one
extension.

$ file ./98/08/15BFE7E6-7590-45C1-81FF-67540D5DC5A8/EBC329CE-AAA9-4302-B3E6-4AB1E9F42647.pluginPayloadAttachment | awk -F: '{print $2}'
ISO Media, MP4 Base Media v1 [IS0 14496-12

$ file ./30/00/7E8B4C97-4F79-4A52-ABA8-0F84588BE91A/B3255B1E-A8E5-419C-A543-C3A07B3DD7B4.pluginPayloadAttachment | awk -F: '{print $2}'
PNG image data, 1024 x 576, 8-bit/color RGBA, non-interlaced

So, getting ALL the attachments will require a lot more work.

The upside is that if I copy those files to "test.mp4" and "test.png"
the files open just fine. The bad news is, hooboy is that going to be
annoying.

I was hoping mdls would be some help, it is not. I think there is an
sqlite3 database lurking around, so I may have a poke at that and see
what fall out.

--
Thanks to the human heart by which we live, Thanks to its tenderness,
its joys, and fears, To me the meanest flower that blows can give
Thoughts that do often lie too deep for tears

[Tim Lance]

On Sep 2, 2020, Lewis wrote

(in article <slrnrkvj44....@ProMini.lan>):

Yes. On my MacBook Pro. I do text on iPhone and iPad. All the files I care
about are family stuff (mostly of grandkids) and we all are Apple exclusive
and do have iCloud active for everything.

>
> This makes no sense, if the app is showing the attachments they must be

> on your computer. Now it gets very crazy. Some back past September 2019 the attachments show in texts have icons to download them. Where it gets crazy is
those files go back to 1918 and so when I click to download they appear
almost instantly. I presume because they are in fact already in the
Attachments folder.

>
>
> There are also files in the folders with the suffix
> pluginPayloadAttachment but I haven't looked into those. I assumed they
> help URLs and previews of those URLs.
>
> Nearly all my message are from other Messages users, and not from SMS
> message. This may make a difference?
>
> > Thanks for sticking with me.

—

t

[Lewis]

In message <slrnrkvkv8....@ProMini.lan> Lewis <g.k...@gmail.com.dontsendmecopies> wrote:
> So, getting ALL the attachments will require a lot more work.

Maybe not so much work.

Here is what I did.

Get all the pluginPayloadAttachment files and there type into a file
$ for file in **/*pluginPayloadAttachment; do file $file >> ~/Desktop/Attachments.txt; done

Open the file in BBEdit (I believe this all works in the free version)
$ bbedit ~/Desktop/Attachments.txt

Then, in BBEdit I first added a line number to each line (about 20,000)
using Text menu => Add/reove line numbers.

I used the text menu "Process lines containing" and removed all the
lines that matched ": empty " or ": MS " (note the spaces) as I did not
care about MS icon files.

Then, I ran a search and replace:

Find: ^ *([^ ]+) ([^:]*): ([^ ]+) (.*)$
replace with: cp -a \2 ~/desktop/Attachments/\1.\3 # \4

The find first captures the line number with any optional leading spaces
from the strart of the line this is \1

^ *([^ ]+)

Followed by a space, and then captures the name of the file (everything
up to the first : character) followed by the colon and a space. This is
\2

([^:]*):

Then, it captures the next set of non space characters (JPEG, ISO, PNG,
etc) this i s \3

([^ ]+)

And then it finally captures the rest of the line (this part is
optional). this is \4

(.*)$

Now, Replace with cp -a <filename> <path/to/folder><linenumber><the
JPEG, ISO, PNG, etc string), a # character, and the rest of the line.

Finally, I search for ISO and change it to mp4 and MS and cahnge iot to
ico.

the lines end up looking like this:


cp -a 02/02/BC1F8714-91D6-48AA-8753-251AB0198389/06F44D03-2DF9-4E81-89BA-E08948F369A7.pluginPayloadAttachment ~/desktop/Attachments/253.PNG # image data, 1024 x 465, 8-bit/color RGBA, non-interlaced

The # is a comment character in bash or zsh, so the rest of the line
after that is ignored by the shell, but it seemed worth keeping around
at least for now.

I can save the file and source it from the commandline, or I can use the
paid BBEdit feature of using a shell worksheet. This has proved to be
useful as there were a very few (less than 20) gif files, a couple fo
"PC" files ("PC bitmap 15x16x24" that I do not care about).

The vast majority of these files are tiny tiny files, and scanning
trough all of them is not worth it for me, so I an starting at the top
and scanning down. I doubt I will look at anything under 100K (90% of
mine). Most of the small ones appear to be favicon files from websites
and glyohs that are renderd as images instead of proper utf-8
characters. Probably stuff I go from Android users?

I may be a tad bit bored and have a wee bit of a problem,. Who can say.

--
Friction can be a real drag.

[Tim Lance]

On Sep 2, 2020, Tim Lance wrote
(in article <RgR3H.241429$7vd....@fx35.iad>):

Hey! A question about dates. I’ve said the stuff in Attachments only goes
back to 2018. That was the last time I did a totally clean install. When
iCloud was turned on and all the old stuff came down. Even if an attachment
was originally texted in say 2015 would the file created date not be 2018
since that is when they were put onto my computer?

I did not actually look through those 600+ images last night.

—

t

[Tim Lance]

On Sep 2, 2020, Tim Lance wrote

(in article <ylR3H.398130$eN2.1...@fx47.iad>):

Nuts. Nope they are all from the last 2 years.

—

t

[Tim Lance]

On Sep 2, 2020, Lewis wrote

(in article <slrnrkvoef....@ProMini.lan>):

Haha.

I think someone needs to develop a full-fledged app.

—

t

[Lewis]

In message <ylR3H.398130$eN2.1...@fx47.iad> Tim Lance <n...@here.org> wrote:
> Hey! A question about dates. I’ve said the stuff in Attachments only goes
> back to 2018. That was the last time I did a totally clean install. When
> iCloud was turned on and all the old stuff came down. Even if an attachment
> was originally texted in say 2015 would the file created date not be 2018
> since that is when they were put onto my computer?

I am not sure, but I would think the dates would be preserved.

> I did not actually look through those 600+ images last night.

View by icon in the finder, set the size slider to large, and just page
down. Should give you a decent idea of what is there.




--
The more you tighten your grip, Tarkin, the more star systems will
slip through your fingers.

[Tim Lance]

On Sep 2, 2020, Lewis wrote

(in article <slrnrkvuqd....@ProMini.lan>):

> In message<ylR3H.398130$eN2.1...@fx47.iad> Tim Lance<n...@here.org> wrote:
> > Hey! A question about dates. I’ve said the stuff in Attachments only goes
> > back to 2018. That was the last time I did a totally clean install. When
> > iCloud was turned on and all the old stuff came down. Even if an attachment
> > was originally texted in say 2015 would the file created date not be 2018
> > since that is when they were put onto my computer?
>
> I am not sure, but I would think the dates would be preserved.
>
> > I did not actually look through those 600+ images last night.
>
> View by icon in the finder, set the size slider to large, and just page
> down. Should give you a decent idea of what is there.

Just flicking through the directory using QuickLook was fine. Nope no old pix
os young pups or baby grandkids. :(

—

t