Word 2016 ignored flavors

[Peter N Lewis]

Hi all,

I figured I should pass this on to all of you - if you do clipboard history, and read the clipboard flavors, Word 2016 has some new flavors you will want to avoid reading, specifically:

com.microsoft.Link-Source
com.microsoft.ObjectLink

If you read them, Word 2016 will do stupid things.

The full list of flavors I currently ignore in Keyboard Maestro is:

dyn.agk8ywxw2ea
dyn.ah62d4rv4gk8ycwndkk
dyn.ah62d4rv4gu8zaxcxmuufa4dtsv1y24psqzggc8pfsk
dyn.ah62d4rv4gk8y2xwpnq
dyn.ah62d4rv4gk8y8wwqmq
dyn.agk8y2xwpnq
dyn.agk8y8wwqmq
dyn.ah62d4rv4gk81g7d3ru
dyn.agk81n65yru
com.adobe.illustrator.aicb
com.adobe.illustrator.hfs
dyn.agk8ynxncnq
dyn.ah62d4rv4gk8ynxncnq
com.microsoft.Link-Source
com.microsoft.ObjectLink

If you are missing any of them, I suggest you add them. If you know of any other flavors that are dangerous to read, I would love to hear about them.

Enjoy,
Peter.


—
Keyboard Maestro <http://www.keyboardmaestro.com/> Editors' Choice Award winner.
Macros for your Mac <http://www.stairways.com/> <http://download.keyboardmaestro.com/>
Join us on the Keyboard Maestro forum <http://forum.keyboardmaestro.com/>

[Thomas Tempelmann]

Thanks for sharing them, Peter.

For those who prefer the OSType and NSPasteboard codes, here's them translated:

> dyn.agk8ywxw2ea

"INX "

> dyn.ah62d4rv4gk8ycwndkk

"AICB"

> dyn.ah62d4rv4gu8zaxcxmuufa4dtsv1y24psqzggc8pfsk

NSPbType: "PLSL PhotoLineLayer"

> dyn.ah62d4rv4gk8y2xwpnq

"LNKS"

> dyn.ah62d4rv4gk8y8wwqmq

"OJLK"

> dyn.agk8y2xwpnq

"LNKS"

> dyn.agk8y8wwqmq

"OJLK"

> dyn.ah62d4rv4gk81g7d3ru

"styl"

> dyn.agk81n65yru

"ustl"

> dyn.agk8ynxncnq

"EMBS"

> dyn.ah62d4rv4gk8ynxncnq

"EMBS"

Thomas

[Peter N Lewis]

com.microsoft.DataObject is also problematic - if you include that flavor, then MS will paste whatever it already has on the clipboard and will ignore the new clipboard you have just set. So it is best avoided as well.

Enjoy,
Peter.

[Thomas Tempelmann]

Peter, have you any oneone else tried to contact MS about this?

It's understandable if they want to use such a shortcut, but they
should make it a private flavor (which, unfortunately, only works with
the old Carbon Pasteboard APIs, I'm afraid) so that other apps don't
accidentally copy/restore it. So, they should be able to fix this
instead of us having to work around their non-standard way of using
the clipboard, right?

Thomas

--
Thomas Tempelmann
Irradiated Software

[Thomas Tempelmann]

Hi all!

After Peter notified us a while ago about some oddly behaving flavors
in the pasteboard, I've finally written to the frameworks-it Apple
list today about this topic and got already responses from two guy
from MS, offering help.

One asks for reproducible steps so that he can file a bug report if
that's appropriate. The other can give explanations why they're doing
it this way and offers to help on better dealing with its effects.
Maybe we can even convince them on adding some flavors in the future
that'll make this easier for us.

For instance, how about defining an new type that lists types that
shall be ignored when recording the entire contents?

Thomas

[Peter N Lewis]

> On 23 Sep 2015, at 00:36, Thomas Tempelmann <tho...@irradiatedsoftware.com> wrote:
> One asks for reproducible steps so that he can file a bug report if
> that's appropriate.

In the Terminal, run the command:

defaults write com.stairways.keyboardmaestro.engine IgnoredClipboardFlavors nothing

Then launch Keyboard Maestro.

That will turn off ignoring any clipboard flavors.

Then copy and paste in Word will behave oddly, and the clipboard history in Keyboard Maestro (Command-Control-Shift-V) will show the weird caching behaviour.

Specifically:

Type some text, say

one two three four five six seven eight

Copy and paste the four to the end, copy and paste the six to the end.

menu Insert ➤ Bookmark shows multiple bogus OLE_LINK entries

Open Keyboard Maestro’s clipboard history. Double click on the “four” to paste it. “six” is pasted instead.

In the Terminal,

defaults write com.stairways.keyboardmaestro.engine IgnoredClipboardFlavors "com.microsoft.Link-Source|com.microsoft.ObjectLink|com.microsoft.DataObject”

(one line, straight double quotes, not curly quotes).

Relaunch Keyboard Maestro Engine (in Keyboard Maestro, File ➤ Quit Engine, File ➤ Launch Engine)

Close and open a new document and repeat the above. Both issues are resolved.

In the Terminal,

defaults delete com.stairways.keyboardmaestro.engine IgnoredClipboardFlavors

just to clean up - the MS flavours and more are included by default.

> The other can give explanations why they're doing
> it this way and offers to help on better dealing with its effects.
> Maybe we can even convince them on adding some flavors in the future
> that'll make this easier for us.
>
> For instance, how about defining an new type that lists types that
> shall be ignored when recording the entire contents?

It’s certainly possible.
Peter.



—
Keyboard Maestro <https://www.keyboardmaestro.com/> Editors' Choice Award winner.
Macros for your Mac <https://download.keyboardmaestro.com/>
Join us on the Keyboard Maestro forum <https://forum.keyboardmaestro.com/>

[Thomas Tempelmann]

Hi all!

It appears that Erik Schwiebert from Microsoft is now with us on this list.

Erik has worked on the clipboard code and explained to me that it's
using these special flavors in the pasteboard along with OLE objects
between the various Office apps. That means they can't be flagged as
private because the other Office apps need to see them.

I suppose we could now do two things:

1. Erik could tell us which currently used pasteboard types are used
privately and should not be recorded and restored by a clipboard
recording app. Erik, is that possible? (Currently, we know of:
com.microsoft.Link-Source, com.microsoft.ObjectLink,
com.microsoft.DataObject).

2. Maybe we can agree to define a special type that apps could use in
the future to declare "private" types that should not be restored by
clipboard recorders, ever. If Erik or someone else at MS can agree to
add such a type in the future, we might also be able to persuade
others such as Adobe, to do the same (Adobe has put problematic types
into the pasteboard in the past, too, I believe).

Would everyone here please state whether they like to participate in
this discussion? If it's only just Peter and me, then we can keep this
a private discussion. I'd appreciate others to at least voice
agreement with or interest in these plans.

Thomas

[Erik Schwiebert]

Hi all,

I was noodling on this the other day with some people at work. We thought that maybe declaring a "org.nspasteboard.privatetypes" flavor, whose contents are a serialized NSData of an NSArray of NSStrings of flavors that pasteboard caching apps should ignore might be a way to go.

As for the specific Microsoft flavors that should never be cached, I think there are more than the three Thomas lists below (code inspection shows a number of "suspect" flavors) but Ill have to dig some more. For Office 2016, they will definitely start with "com.microsoft.", as Office 2016 is now using NSPasteboard APIs and real Uniform Type Identifier strings. As Peter noted on the group site, Office 2011 uses a number of "dyn*" names because Office 2011 is still using the ancient Scrap Manager / ResType APIs. The Office 2011 code is not going to change so whatever you have deciphered for that is probably fine.

Thoughts?
Schwieb

[Greg Scown]

Hi Thomas, and All,

I hope you'll continue to have the conversation with Peter and Erik publicly on this list. TextExpander isn't a clipboard manager, so I don't have much to add to the conversation, but I do have an interest in seeing how it plays out, then figuring out how to adjust what is written on NSPasteboard.org to reflect the wishes of the group and updating TextExpander to match. Put another way, just because I'm not contributing directly doesn't mean I'm not listening intently.

Thanks!

==> Greg

Greg Scown

gr...@smilesoftware.com

Twitter: @macgreg

[Thomas Tempelmann]

On Wed, Sep 30, 2015 at 6:48 PM, Erik Schwiebert <eri...@microsoft.com> wrote:
> We thought that maybe declaring a "org.nspasteboard.privatetypes" flavor, whose contents are a serialized NSData of an NSArray of NSStrings of flavors that pasteboard caching apps should ignore might be a way to go.

Yes on the flavor! I'm relieved to hear that the issue is actively
considered and discussed over at the Mac Office team :)

Though, I'd rather keep the data format as simple as possible. As
hopefully no one comes up with type names that are using control chars
or something else exotic, I'd rather see the contents being plain text
lines, UTF-8 encoded. That would also make it easier to parse them by
eye (e.g. with the various clipboard showing tools).

One thing we need to define is which type of names we'll use. With the
Carbon based Pasteboard APIs, we get to use UTIs, whereas NSPboard
types use a different kind of names, which I suspect being based on
NeXT type names. While one type of names can be converted to the other
with some API functions, with the pasteboard functions even doing some
conversions automagically (or so it seems to me), sometimes it needs
to be very clear with one it is.

So, if we come up with a "org.nspasteboard.privatetypes" flavor, we
need to make it clear that they're using UTIs, not NeXT style names. I
wonder if naming the flavor "org.nspasteboard.privateUTIs" is the
better name for this purpose, avoiding misinterpretations.

However, I am not sure if that works for everyone. Maybe we actually
need TWO flavors, one for pricate NSPboard names and one for UTIs?
Does someone have any input on this?

[Günther Blaschek]

Hi,

@Erik: Welcome to the list.

@Thomas: Yes, please keep the discussion public. Even if someone is not directly affected, it will still be interesting to learn about the conclusions coming out of this discussion.

@all:

I am not sure that the current workaround (ignoring critical flavors) really solves the problem. This means that any utility that saves and restores clipboard contents destroys the information on the pasteboard that was put there deliberately by an Office component.

For example, consider the following scenario: You copy something in Word and then paste that into Excel. This will cause Word to keep the promise, so Word knows that Excel has picked up the clipboard contents.

If some third-party utility (could be a clipboard historian or an expander like Typinator or TextExpander) removes the special flavors between the Copy and the Paste, the internal communication between the Office components would break because a subsequent Paste in Excel wold no longer "see" the magic flavors.

It does not matter whether we agree on a list of taboo flavors or the Office team provides additional information on the pasteboard. The latter solution is more elegant and extensible, but it still breaks the communication between the Office components.

@Erik: Can you tell us more about the purpose of these "magic flavors"? Which technical problem do they solve?

Maybe we can find a different solution that solves the problem in a different way.

If the purpose of these flavors is that the target application (in which the Paste takes place) tells the source application that the copied data is being used, a system-wide notification might do the trick. In the example above, the fact that some application had fetched the magic flavors would not yet trigger an action in the source application. All historians and utilities that save/restore the clipboard could fetch and restore the magic flavors as often as they like, and nothing bad would happen. But when Excel picks up the clipboard, it would send a notification that tells Word (or PowerPoint, or any other application that needs to know) that the clipboard's contents are being used in a special way.

However, without knowing more about the purpose, this is just a wild guess.

Cheers,

gue

--

gue = Günther Blaschek

[Thomas Tempelmann]

On 01.10.2015, at 09:38, Günther Blaschek <gue...@gmail.com> wrote:
>
> For example, consider the following scenario: You copy something in Word and then paste that into Excel. This will cause Word to keep the promise, so Word knows that Excel has picked up the clipboard contents.
>
> If some third-party utility (could be a clipboard historian or an expander like Typinator or TextExpander) removes the special flavors between the Copy and the Paste, the internal communication between the Office components would break because a subsequent Paste in Excel wold no longer "see" the magic flavors.

I don't think so - provided that Office does use the Pasteboard as it's intended to be used:


A clipboard recorder would remember only the non-internal flavors. When pasting them back, the receiving Office app would take the contents as if they were created by another app. The only disadvantage in this process is that the performance will suffer from this process.

In other words:

With the internal Office flavors present, copying data between Office apps is very likely more efficient, because there will be no need to convert (export & import) the data from Office's internal structures into Mac-friendly structures.

With the clipboard recorder removing those internal flavors, the Office apps will be forced to perform that extra data conversion, though, meaning it'll be slower, and - at worst - even lose some critical information.

Erik, am I correct in my assumptions?

(This leads me to another issue which we might solve along: The goal would be to avoid multiple redundant conversions when storing the clipboard contents, e.g.: While Excel can render a PDF when asked to deliver that flavor's data, it will be a fairly slow process, and such a rendered flavor is superfluous for preserving Excel's clip contents - it would be smarter to only preserve an internal flavor that Excel can import/export without loss. More on this later, once Erik has given us some clarifications.)

Thomas

[Günther Blaschek]

Sorry, Thomas, but no. This is about breaking an Office feature, not about performance.

Let me explain this in more detail. I'll use an example that you can try yourself, if you have Office 2016.

Preparation:

Open new Word and Excel documents.

In Word's preferences, click "View" and make sure that "Bookmarks" are turned on in "Show in Document".

Enter some text in Word, from which you will copy a text fragment.

Copy/paste the text a number of times, so you have multiple copies for repeating exactly the same experiment in different situations.

In my case, I entered this text:

First, make sure that nothing interferes with Copy/Paste: Select the text as shown above and press command+C.

If it remains as shown, everything is OK.

It can be that brackets appear around the text, like this:

This is a sign that some other component on your Mac has picked up the "magic flavors". As a result, Word now incorrectly believes that some other application has pasted these three words as a "Microsoft Word Document Object", so it marks the source object as a "bookmark".

If this happens as a side effect when you just copy a text fragment, you have some clipboard historian utility running that fetches the magic flavors immediately after each clipboard change.

I could reproduce this issue with Collective 2.0.2 (which does not yet avoid the new flavors of Word 2016), so I had to disable that.

Back to the experiment: If you copy the text fragment in the word document and do NOT get the brackets, everything is OK, and you can continue with the next step:

Switch to Excel, then select Edit / Paste Special…

In the list of formats, select "Microsoft Word Document Object" and click OK.

Results:

a) a new text object that floats above the spreadsheet:

b) bookmarks brackets around the text fragment in the word document, as shown above.

Word now knows that the text had been copied to Excel. I'm not quite sure what this means, but I assume that the Word document is now somehow linked to the Excel document. (Erik, can you clarify?)

Now the same experiment again, with the latest version of Typinator (currently 6.7b5). You can get it here:

http://www.ergonis.com/downloads/beta/

The last official release 6.6 does not yet know about the new flavors in Office 2016, but the current beta version does.

If you read this when beta testing has been closed, get the final version 6.7 from here:

http://www.ergonis.com/downloads/

(You may be able to reproduce this with other utilities as well, but I am not aware of their current status, so I cannot tell for sure which version of which tool will ignore the new flavors.)

In Typinator, create an abbreviation with a "Formatted Text" expansion. The actual text does not matter. But it is important to use a "Formatted Text" expansion, to make sure that Typinator needs to use the clipboard, which involves saving and restoring the clipboard.

Variation of the experiment:

Copy text in the Word document.

Before pasting in Excel, type the Typinator abbreviation.

As above, switch to Excel, do a "Paste Special…" and insert a "Microsoft Word Document Object".

In Excel, the object will appear as before.

But when you return to Word, the text fragment now appears WITHOUT BRACKETS.

The reason for this is that Typinator removed the "magic flavors" from the pasteboard as the result of the save/restore.

Word no longer "sees" that Excel has fetched the data.

I don't know exactly what this means, but I suppose that the link between Word and Excel is now broken.

Whatever we, there is always the risk of unwanted results:

• When a third-party utility fetches and restores the magic flavors, Word creates a bookmark of previously copied text, even though it has not been pasted as a "Microsoft Word Document Object".
• If a third-party tool ignores the flavors when saving and restoring the pasteboard, Word no longer gets notified that a text fragment had been pasted as a "Microsoft Word Document Object".

I don't know which is worse.

@Erik: Can you confirm my assumptions?

Cheers,

gue

[Thomas Tempelmann]

It can be that brackets appear around the text, like this:

This is a sign that some other component on your Mac has picked up the "magic flavors"

 

Ah, the good old Object Containers. Reminds me of OpenDoc! :)

Okay, so it appears that when you request certain flavors from the pasteboard, Office turns the selection into a container that will then act as a reference. I guess that when you paste such an object from Word into Excel, and then later change it in Word, the change will also appear in the Excel, right?

That may actually be useful and desired, so I wonder if Office can make that work with clipboard recorders as long as the objects still exist. For example: I copy from Word first the text "THREE", then the text "WORDS". Now I use my recorder to paste the first copied item into Excel or into another Word document, as a (OLE?) Object. Can that be made to work, Erik? What Peter told us originally in this thread, at least one Office flavor seems to always assume that the pasted item is the LAST item copied, so that pasting the object would lead to pasting WORDS, not THREE - but can the object for THREE be made persistent by a recorder somehow (provided that the document from which it origins is still alive)?

Thomas

[Erik Schwiebert]

So the logic behind the com.microsoft.* flavors is pretty complex and unfortunately is predicated on clipboard and OLE data sharing design on Windows. They don’t really obey the normal rules for Mac clipboard use, which is why you’ve seen problems when you archive them.  Theese flavors are intended for data sharing between Office apps only. It is completely safe for clipboard archivers to ignore them, and better to do so. Thomas’s assumptions below are close-enough for this purpose. Obviously, the clipboard archiver should call in all the promises for non-private flavors so that the data can be archived (archiving a promise is no good once the promising app exits).  If a clipboard archiver provides a cached clipping to an Office app, having stripped the private flavors, then Office should treat it as any regular paste from a non-Office source.

Unfortunately, there’s not really an easy way to render just the minimum Excel-internal format for use for later consumption, and what Excel needs is different from Word, or from PowerPoint, etc.  

I actually wrote a clipboard archiving feature for Mac Office back in 2002-2003 that shipped as part of Office 2004 (and was cut from Office 2011), so I’m familiar with the complexities involved here. For that feature we had to be careful to not archive the temporally-dependent data and to make sure we did call in all promises to cache that data. SWhen our apps promise a multitude of flavors, it’s because we don’t know what format the receiving app will want (unicode text, text + styles, image, etc). We promise all and render only what the receiving app asks for. When archiving a clipping, you have no idea what flavor the consuming app will want later (next hour, day, week or a year later) so you have to archive them all.

I don’t want to bikeshed too much on the design of a flavor to mark private data. Human readability is nice, but so is ease of code to generate the data. Our current code would find it very easy to generate an NSArray of NSStrings, but it’s not a lot more work to dump that to a blob of ASCII text separated by newlines. :)  Any other proposals out there?

Schwieb

[Peter N Lewis]

Sigh, just reviving this yet again, it seems somewhere along the lines the flavors Microsoft uses changed to the dyn versions:

dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e82xnqzv1kxdmr3zu (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.ObjectLink)
dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e24psrq0zg55zsmv0n (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.Link-Source)

So if you exclude com.microsoft.ObjectLink and com.microsoft.Link-Source to avoid the obtuse "Bookmark" behaviour in Word, consider adding the dyn variants as well.

Regards,
Peter.

--

[Günther Blaschek]

Hi Peter,

Thanks for the information.

Is this related to the clipboard history issue that was reported by ajg23 in the forum?

Regards,

gue

[Thomas Tempelmann]

Thanks, Peter.

Just had a customer contact me saying "pasting does not work any more - it always pastes just the latest item". Had no idea what's going on there until he sent me a screen recoding - and he's been using Word. Then it clicked.

Having learned from past issues, I do not only hard-code those exceptions in the app but also read them from the prefs file. So I can now simply send the affected users a command to update their prefs to fix this issue right away.

Thomas

[Thomas Tempelmann]

dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e82xnqzv1kxdmr3zu
dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e24psrq0zg55zsmv0n

BTW, these UTIs resolve to the NSPasteboard names com.microsoft.ObjectLink and com.microsoft.Link-Source.

Looks like a programming mistake to me, where a programmer confused UTIs with NSPasteboard identifiers, maybe when updating the code to use newer APIs. Or maybe it was done on purpose.

[Peter N Lewis]

And a new one in Word 15.40:

dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1ek2pyqfh0e4xfqr4a (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.DataObject)

If you copy that flavor, Word will never let you restore the clipboard to that entry.

Regards,
Peter.

On 23 Oct 2017, at 12:25, Peter N Lewis <pe...@stairways.com.au> wrote:
> Sigh, just reviving this yet again, it seems somewhere along the lines the flavors Microsoft uses changed to the dyn versions:
>
> dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e82xnqzv1kxdmr3zu (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.ObjectLink)
> dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1e24psrq0zg55zsmv0n (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.Link-Source)
>
> So if you exclude com.microsoft.ObjectLink and com.microsoft.Link-Source to avoid the obtuse "Bookmark" behaviour in Word, consider adding the dyn variants as well.
>
> Regards,
> Peter.
>
>

>> On 30 Sep 2015, at 21:56, Thomas Tempelmann <tho...@irradiatedsoftware.com> wrote:
>>

> --

> Keyboard Maestro <https://www.keyboardmaestro.com/> Editors' Choice Award winner.
> Macros for your Mac <https://download.keyboardmaestro.com/>
> Join us on the Keyboard Maestro forum <https://forum.keyboardmaestro.com/>
>

--

[Thomas Tempelmann]

On Tue, Dec 12, 2017 at 3:38 AM, Peter N Lewis <pe...@stairways.com.au> wrote:

And a new one in Word 15.40:

dyn.ah62d4qmxhk4d425try1g44pdsm11g55gsu1ek2pyqfh0e4xfqr4a (?UTTypeConformsTo=13:com.apple.nspboard-type=com.microsoft.DataObject)

Yes. And this was already true for 15.39 in my case.

Obviously, since the old IDs you had to exclude were

  com.microsoft.ObjectLink, com.microsoft.Link-Source and com.microsoft.DataObject, 

the new "dyn." names, created by using them with the new API, have all to be excluded as well, not just the two you mentioned first. I had been aware of this but should have mentioned it on this list right away. Sorry.

Thomas

[Erik Schwiebert]

Hi all,

 

Sorry I missed this thread earlier. This is an intentional design change – we are now registering our pasteboard identifiers with the OS to declare proper UTI conformance, which is necessary to support drag and drop where the only data on the drag pasteboard is one of our custom types.

 

You’ll have to add sniffing for these new dynamic types. They shouldn’t change so that should be safe to do.

 

Schwieb

 

From: nspast...@googlegroups.com [mailto:nspast...@googlegroups.com] On Behalf Of Thomas Tempelmann
Sent: Tuesday, December 12, 2017 12:06 AM
To: nspast...@googlegroups.com
Subject: Re: Word 2016 ignored flavors

 

On Tue, Dec 12, 2017 at 3:38 AM, Peter N Lewis <pe...@stairways.com.au> wrote: