Issue: Twitter #hashtags to TiddlyWiki Tags -- HOW, safely

[@TiddlyTweeter]

Ciao tutti,

I have an issue I can't solve. It may be I'm looking at the problem the wrong way because of my dearth of technical understanding. Below are notes laying out my understanding to date. I hope they are clear. Please ask if not ...

  Use Case: I write lots of Twitter posts in TiddlyWiki. Twitter is a plain text format so they are always written in that Type in TW. Twitter uses #hashtags. To use TW optimally as a partner for Twitter the #hashtags I write in a "tweet" tagged Tiddler I want to automatically add as Tiddler TW Tags.

  Background: Riz, some many months back, devised a way to do this that hooks into the save system--though it was meant more as a "proof-of-concept" than a finished workable solution. It can be found here.

  The Problem: The issue is that by hooking into the fundamental save mechanism Riz's solution means that ANY Tiddler with a "#" in it gets processed on save and that can produce zillions of unwanted weird "tags". Basically its in conflict with (1) WikiText, (2) Stylesheets, (3) and anywhere else you might use a "#" / "pound" sign. The problem was discussed in some detail before in this thread.

  Possible Solutions: (1) Limit the action on save (in Riz's experimental saver) so that the #hashtag to Tag phase of it ONLY runs IF (a) the Type is "plain/text", AND (2) the Tiddler concerned is Tagged "tweet".  (2) Some other solution I'm not thinking of.

  Notes: It is NOT necessary that the addition of TW Tag versions of the #hashtags contained in the text of the Tiddler happens on save. It would be just as functionally effective to have a macro that could be periodically run (on a click) that finds ALL Tiddlers Tagged "tweet" of Type "plain/text", performs the magic on them, and leaves aside all others.

Any thoughts or solutions would be much appreciated.

Best wishes
Josiah 

[Mark S.]

Now that Riz has done the hard work of coming up with the clever tag-extraction code, it's pretty easy to to clone it and combine it with the existing save code.

If you copy the following over the existing code in $:/core/ui/Buttons/save , then save, and reload the tag-extraction process will only happen if the tiddler is tagged "Tweet" and is of type text/plain. As a side effect (feature?) it won't let you save at all if you add a tweet tag and forget to change the type to text/plain.

There may be some other logic flaw, since the logic of the two filters is not a precise inverse. In any event, be sure to make a back-up before trying anything.

Mark

<$list filter="[all[current]tag[Tweet]type[text/plain]]">
<$set name=test filter="[list[!!text]prefix{$:/_state/tesseract/hashtag2tag}] +[removeprefix{$:/_state/tesseract/hashtag2tag}]">
<$fieldmangler><$button tooltip={{$:/language/Buttons/Save/Hint}} aria-label={{$:/language/Buttons/Save/Caption}} class=<<tv-config-toolbar-class>>>
<$action-sendmessage $message="tm-add-tag" $param={{$:/temp/NewTagName}}/>
<$action-listops $tags=<<test>>/>
<$action-deletetiddler $tiddler="$:/temp/NewTagName"/>
<$action-sendmessage $message="tm-add-field" $name={{$:/temp/newfieldname}} $value={{$:/temp/newfieldvalue}}/>
<$action-deletetiddler $tiddler="$:/temp/newfieldname"/>
<$action-deletetiddler $tiddler="$:/temp/newfieldvalue"/>
<$action-sendmessage $message="tm-save-tiddler"/>
<$list filter="[<tv-config-toolbar-icons>prefix[yes]]">
{{$:/core/images/done-button}}
</$list>
<$list filter="[<tv-config-toolbar-text>prefix[yes]]">
<span class="tc-btn-text"><$text text={{$:/language/Buttons/Save/Caption}}/></span>
</$list>
</$button>
</$fieldmangler>
</$set>
</$list>
<$list filter="[all[current]!tag[Tweet]]">
<$fieldmangler><$button tooltip={{$:/language/Buttons/Save/Hint}} aria-label={{$:/language/Buttons/Save/Caption}} class=<<tv-config-toolbar-class>>>
<$action-sendmessage $message="tm-add-tag" $param={{$:/temp/NewTagName}}/>
<$action-deletetiddler $tiddler="$:/temp/NewTagName"/>
<$action-sendmessage $message="tm-add-field" $name={{$:/temp/newfieldname}} $value={{$:/temp/newfieldvalue}}/>
<$action-deletetiddler $tiddler="$:/temp/newfieldname"/>
<$action-deletetiddler $tiddler="$:/temp/newfieldvalue"/>
<$action-sendmessage $message="tm-save-tiddler"/>
<$list filter="[<tv-config-toolbar-icons>prefix[yes]]">
{{$:/core/images/done-button}}
</$list>
<$list filter="[<tv-config-toolbar-text>prefix[yes]]">
<span class="tc-btn-text"><$text text={{$:/language/Buttons/Save/Caption}}/></span>
</$list>
</$button>
</$fieldmangler>
</$list>

[@TiddlyTweeter]

Ciao Mark S.

Thanks so much for that! Magic stuff. I'll test it a couple of days to check there are no issues.

A couple of initial comments ...

- For others interested. For this to work, in addition to Mark S. improved code--that you overwrite "$:/core/ui/Buttons/save" with--one needs create a Tiddler called "$:/plugins/tesseract/hashtag2tag/settings" and put the character "#" (or any other leading marker you want) in it. If you don't do that every word would become a tag.

- I noticed, as you said, that if you don't set the type to "text/plain" when you tag a Tiddler "Tweet" then there is no save button. All to the good :-). In actual use in TW I want this for I'd never see that because I have a button that creates "Tweetable Tiddlers" pre-tagged "Tweet" and of type "text/plain".

There is further work needed on this. Particualarly how the in-text #hashtagged words get processed. At the moment the gizmo doesn't know that "#Buddha's" "#Buddha" & "#buddha" should ALL be converted to TW Tags as "#buddha" (Twitter is not case sensitive; punctuation shouldn't be in the TW tag). But before bothering you and others about this I'll try and work out how to do it myself.

When its all finished I'll contact Riz and see if he wants to update his experimental plugin. The method you have given teeth to has applications beyond Twitterphiliacs.

Again, many thanks
Josiah

[Mark S.]

There is further work needed on this. Particualarly how the in-text #hashtagged words get processed. At the moment the gizmo doesn't know that "#Buddha's" "#Buddha" & "#buddha" should ALL be converted to TW Tags as "#buddha" (Twitter is not case sensitive; punctuation shouldn't be in the TW tag). But before bothering you and others about this I'll try and work out how to do it myself.

Riz's code uses existing filter operators. To make the code work like you suggest it will take either a new filter operator and/or a new javascript macro. I can imagine a lower and upper filter operator. Handling the apostrophe is something else. Maybe if the user could just do "#Buddha 's"

Mark

[@TiddlyTweeter]

Ciao Mark

That's helpful. When writing Twitter posts I could likely, without loss of meaning, avoid apostrophes on hashed words by changing sentence order. But changing orthography just for TW feels like a step too far. Its an interesting issue.

Best wishes
Josiah

[Mark S.]

I just realized that if all you want is remove the suffix "'s" that that can be done right now by changing the first filter expression to:

[list[!!text]prefix{$:/_state/tesseract/hashtag2tag}] +[removeprefix{$:/_state/tesseract/hashtag2tag}removesuffix['s]]

If there's other special grammatical deconstructions, you might try to list them out.

Mark

[@TiddlyTweeter]

Ciao Mark S.

Brilliant thought! I can't think of many use cases beyond that one on added punctuation in English I'd use. Maybe, once in a blue moon, I'd write a hashed word followed by a colon "#Buddha: A man under a tree". A more common one might be a full-stop: "And they came to the #Buddha." I think the point you are making is a great one. One doesn't necessarily need a complex regular expression. (Though the truth is the JavaScript RegEx for these case is really simple---"#\w+\b"---that excludes all unwanted punctuation as \b is a "null-space" match for "word-boundary". I just don't know if TW macros can handle it.)

Best wishes
Josiah

[Mark S.]

AFAIK, there's only the regexp operator, which doesn't remove or add suffixes or parts. So, using baked-in components you'll need removesuffix[].

Mark

[@TiddlyTweeter]

Ciao Mark

I think I should look at what the regexp operator can do. If it uses the underlying JavaScript regex it might work because IF you can get a precise match you don't need add or subtract anything, you just need to return the match.

Josiah

[TonyM]

Josiah,

Just thinking about what you are doing and was wondering if each tweet is a paragraph and #tags are at the end of a paragraph then a #tag will always have a leading space, not be at the beginning and be unlikely to be confused with the other uses of #.

It would seem to me that when you close a tiddler a macro on the view template could test for the tweet tag, extract the hash tags and populate a field from which you can click and create non existing #tag tiddlers as desired and present a tweet button to export in a Twitter standard if desired.

I am not a tweeter myself but I did create a custom export function to export operating system commands to be executed by a loop batch outside the browser.

Just some thoughts. I may be way off but it seems to me processing/parsing a tiddlers content on closing a tiddler could prove useful like detecting and [[ boxing ]] any text that matches an existing tag and more would be cute.

Tony

[@TiddlyTweeter]

Ciao TonyM

Thanks for the post. Because of the awful way that Google Groups work you are probably only dimly aware that the last 6 months have seen enormous developments for Twitter enthusiasts in TiddlyWiki.

Twitter is the one social network that is near comprehensively supported by TiddlyWiki now. Jeremy Ruston made a comprehensive plugin for Twitter timelines. Thomas Elmiger & Riz made, between them, great posting tools. Thomas centred on being able to Tweet SECTIONS of text in normal Tiddlers. Riz centred on being able to post WHOLE Tiddlers.

The issue in this thread is over taming a neat trick Riz came up with that hooks into the save system, such that on-save plain text Twitter style #hashtags get added as TW Tags. Its still not quite perfect, but, thanks to Mark S., its getting closer to proper functioning. And he's made it safe to use, which it wasn't before.

The USER issue, for me, at least, is to enable me to write in my Twitter fashion--straight text-- hashtags wherever I want them, no thought about TiddlyWiki during those moments. Just type & post.

Its being able to do that that makes it a really good posting tool for Twitter.

AND being able to organise stuff after posting, that TW enables, is a godsend because it can give you emergent organisation that is impossible on Twitter itself. I'm already using it, for instance, to produce lists of past Tweets worth repetition that I harvest from TW via filters and then place into inexpensive commercial auto-posting systems, fully ready formatted for them.

----

Coming back to your post ... I think the one thing of wider significance on the "Twitter saga" for TW is how it uses encoded URLs to DO things.

In your case some of the "file initiations" you play with engage with that too? Yes? And it will remain so in the future too--even with the increasingly restrictive developments in browsers. I mean, its not as if you can stop URLs :-).

The other point is about INITIATION of actions on save that this particular thread touches into. I'm not techie enough to know the scope of what could be done, but Riz's gizmo is very suggestive of possibilities.

Best wishes
Josiah

TonyM wrote:

... it seems to me processing/parsing a tiddlers content on closing a tiddler could prove useful like detecting and [[ boxing ]] any text that matches an existing tag and more would be cute.

[Mark S.]

Darn. There's a logic error in the prefix technique I suggested. Don't use it (if you've started).

Mark

[@TiddlyTweeter]

Ciao Mark

Thanks for letting me know. Not to worry. I haven't done anything yet. FYI I have been looking through the "back catalogue" of TW trying to find any extensions people may have made around regular expressions to see if there is anything that could make the code for the processing of the "hashtags" into TW tags less cumbersome.

Best wishes
Josiah

Mark S. wrote:

[Mark S.]

Unfortunately, the "removesuffix" filter does 2 things, rather than just the convenient one thing I wanted. It removes the suffix AND it filters. So though it would work great with #Buddha's it would fail with "#Buddha" .  Correcting for this results in a more complicated filter. Now the logic results in this ginormous filter:

[list[!!text]prefix{$:/_state/tesseract/hashtag2tag}removeprefix{$:/_state/tesseract/hashtag2tag}] [list[!!text]prefix{$:/_state/tesseract/hashtag2tag}removeprefix{$:/_state/tesseract/hashtag2tag}removesuffix['s]] +[!suffix['s]]

Test with caution,
Mark

[@TiddlyTweeter]

Ciao Mark S.

Thanks a lot for the help. I've now got something that's definitely in the right direction. FYI, I made some decisions ...

   The #hashtag prefix I will keep in the TW tags too, so it doesn't need to be clipped off. I don't see any issues doing that? It will better help me to know what is a Twitter related tag and what is purely for internal TW use. That may later also help me construct searches of Twitter.

   I decided to "hardcode" the "#" "prefix" as this gizmo is now so customised for Twitter it makes little sense to have a separate Tiddler holding it. It also helps the code be a more readable.

   I looked back at my Twitter writing style over the last year and saw that there are a LOT of situations where, for TW Tags, I need to snip off punctuation from my Twitter #hashtaggery. The blending of punctuation with hashtags is common on Twitter--mainly because you only have 140 characters to play with. Also Twitter knows what is the hashtag & what is punctuation or glyps abutting it. TiddlyWiki doesn't, so, to be real to my Twitter writing I'll have to do a fair amount of the "removesuffix/prefix stuff". Here's the current assemblage (seems to work!).

<$set name=test filter="
[list[!!text]prefix[#]]
[list[!!text]prefix[#]removesuffix['s]] +[!suffix['s]]
[list[!!text]prefix[#]removesuffix[.]] +[!suffix[.]]
[list[!!text]prefix[#]removesuffix[...]] +[!suffix[...]]
[list[!!text]prefix[#]removesuffix[,]] +[!suffix[,]]
[list[!!text]prefix[#]removesuffix[;]] +[!suffix[;]]
[list[!!text]prefix[#]removesuffix[:]] +[!suffix[:]]
[list[!!text]prefix[#]removesuffix[!]] +[!suffix[!]]
[list[!!text]prefix[#]removesuffix[?]] +[!suffix[?]]
[list[!!text]prefix[#]removesuffix[--]] +[!suffix[--]]
[list[!!text]prefix[...#]removeprefix[...]] +[!prefix[...]]
[list[!!text]prefix[--#]removeprefix[--]] +[!prefix[--]]
">

   Test data for the above ...

This is about #buddha1 #buddha1 #buddha1 who went to #Buddha2's 
concerning #buddha3. who spoke to #buddha4, about the state of 
#buddha5; #buddha6: and questioned #buddha7! and #buddha8? whether
#buddha9-- had a clear conscience about #BUDDHA10... and 
--#Buddha11 as well as ...#Buddha12 #BuddhaDANGER "

It worked! But I haven't learnt yet how to deal with ... #BuddhaDANGER". Double quotes look dangerous if they get assessed for a Tag. But I haven't worked out yet if you can escape them in "removesuffix". They break the script if you use double-quotes straight. Solving that is a bit of finesse I'd like to get to.

But, essentially, its all working pretty well so far. What all this brought up for me was how convoluted it is though. A normal JavaScript regular expression could suck out the exact string you need to transfer to a TW tag without have to think about loads of cases you have to individually address.

Now the next step is for me to work out how to force all the TW tags created to be LOWERCASE :-).

[Mark S.]

Don't use previous version of tolower. For some reason, titles are not always perceived as strings.

New version attached. Always make backups before trying.

Thanks,
Mark


On Tuesday, August 22, 2017 at 4:41:28 PM UTC-7, Mark S. wrote:

You can try this. Import. Save. Reload. Now you can can add tolower[] to each of your filters.

Good luck,
Mark

[@TiddlyTweeter]

Ciao Mark

Thanks. The last version seemed to work. I didn't see any issue. But I'll update.

Molto grazie!
Josiah