[IDEA]: Drag and Drop a Bookmark.html Exported from Chrome/Edge/Firefox

[Mohammad Rahmani]

• Using a modern browser like FF/Chrome/Edge/Opera it is very simple to create bookmarks to the page one visits
• Browser adds link/page title/date added/favicon, ... to the bookmark database
• One can simple export the bookmarks to a .html file
• The type of exported bookmark is <!DOCTYPE NETSCAPE-Bookmark-file-1>
• The file contains entries in the form of <DL>..<DT> which parsing it is not difficult
• Dropping an exported bookmark.html in Tiddlywiki a tiddle with type html is created and one can see the links

I have attached a sample bookmark.html created in FF, drag and drop into https://tiddlywiki.com/prerelease/ see how it works!

Questions and idea

1. How one can create a dropzone to parse the bookmark.html and creates a tiddler per bookmark
2. The link, page title, dates (added/accessed/modified), favicon (in form of data:image/png;base64), ,... can be added as tiddler fields

This way one can surf the web, create his/her bookmark as usual using the browser bookmark manager, export to html and import into Tiddlywiki!

Please share your ideas/solutions/opinions!

Best wishes

Mohammad

[PMario]

Hi,

With FireFox bookmarks can also be exported as a JSON file. ... The structure is a bit weird ... But there should be a description somewhere.

-mario

[PMario]

Hi,

There is an API, to manage and manipulate bookmars with web-extensions.

So it should be possible to "manipulate" them from TW directly. ... But this would be a completely new AddON.

It looks interesting. .. The only problem. .. The day has only 24 hours :/

-mario

[Mohammad Rahmani]

On Fri, May 21, 2021 at 7:51 PM PMario <pmar...@gmail.com> wrote:

Hi,

There is an API, to manage and manipulate bookmars with web-extensions.

So it should be possible to "manipulate" them from TW directly. ... But this would be a completely new AddON.

This sounds great!

It looks interesting. .. The only problem. .. The day has only 24 hours :/

No problem, we still have 7 months until the end of 2021 ;-)

 

-mario

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1171c79c-9d7f-425a-bcd2-a4d974196d8an%40googlegroups.com.

[TiddlyTweeter]

This is for both PMario & Mohammad ...

PMario wrote:

There is an API, to manage and manipulate bookmars with web-extensions.

So it should be possible to "manipulate" them from TW directly. ... But this would be a completely new AddON.

Ha! Interesting. What is the advantage of that API over BRUTE PARSING? 

I looked at the output from Chrome & Firefox. The "href" bits look simple to capture. 

BUT the format also includes icon data sometimes. 

That is more API territory than simple brute parsing a link?

My question: WHO needs the icon bits & WHY? in TW?

For purpose I'd like to know WHAT (attribute="value") end-users need preserving from browser bookmarks???

Just a question

TT

[Mohammad Rahmani]

TT,

 I saw the bookmark.html, I think title and url are minimum, date is also good to be extracted! icon is an extra field here!

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/85b9a103-0bc2-4827-bf29-320e8e7b35c1n%40googlegroups.com.

[Saq Imtiaz]

@Mohammad attached is a very basic attempt at a deserializer to parse a bookmarks file. I wonder if the format is consistent enough amongst browsers.

To test, rename your bookmarks file to have the extension .bookmarks and then import.

Back up your wiki... I barely spent 5 minutes on this so there is no error handling :D

[Saq Imtiaz]

One of the things to consider is that neither title nor URL are bound to be unique in the bookmarks file, so you would need to guarantee somehow that the title used for each tiddler is unique or some will go missing.

[TiddlyTweeter]

PMario ...

Capisco la tua dedizione nel fare la cosa giusta. La vita è breve. Se fossi Dio, allungherei la tua vita. :-)

Allora

TT

[Mohammad Rahmani]

Hi Saq,

 Thank you! Let me give it a try!

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8ab10072-d5e8-4e74-9982-bdd514fa1b79n%40googlegroups.com.

[Mohammad Rahmani]

Hi Saq,

 It Works like a charm! I love it!

Hooora! Hooora!

One minor comments

1. see the attached file! It was generated by Exporting Bookmarks from Chrome!

2. it has only one bookmark

3. the title is like <A ....>AIChE | The Global Home of Chemical Engineers</A>

4. it seems the deserializer has problem with pipe |

4.1. on import I changed the name using the $:/Import UI to  AIChE - The Global Home of Chemical Engineers

4.2. and it works

Best wishes

Mohammad

On Fri, May 21, 2021 at 11:38 PM Saq Imtiaz <saq.i...@gmail.com> wrote:

One of the things to consider is that neither title nor URL are bound to be unique in the bookmarks file, so you would need to guarantee somehow that the title used for each tiddler is unique or some will go missing.

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c0625353-feea-48bf-b4c4-519cb7c831e1n%40googlegroups.com.

[Mohammad Rahmani]

@TT, @PMario

Have a look at https://addons.mozilla.org/en-US/firefox/addon/tiddlymarker/

It only works on FF, but it is very powerful! It supports Node.JS TW and also you can ask to save on local disk as JSON!

Then TW simply read them!

@PMario

The JSON exported by FF bookmark manager does not work in TW! It seems TW cannot read exported JSON!

Best wishes

Mohammad

On Fri, May 21, 2021 at 11:12 PM TiddlyTweeter <Tiddly...@assays.tv> wrote:

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/85b9a103-0bc2-4827-bf29-320e8e7b35c1n%40googlegroups.com.

[Mohammad Rahmani]

Hi Saq,

I tried your solution with FF, Chrome, and Edge! Fortunately it works for all of them! It reads all bookmarks.html generated by those browsers (using Export)

I did not check with Opera!

So, I think it is worth improving! It is very handy as you can export all the bookmarks at once and import to Tiddlywiki!

I found procedures to export from Chrome/Edge to only one folder! So you can customize what is exported and imported to TW, if you are willing to share with others!

The only issue by now is the pipe character where deserialization fails!

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8ab10072-d5e8-4e74-9982-bdd514fa1b79n%40googlegroups.com.

[Mohammad Rahmani]

Saq,

 I think the illegal letters by TW need to be scrapped e.g | [ ] { }

[Saq Imtiaz]

Hi Mohammad,

I think it is time for you to try your hand at some JavaScript :)

This is a good opportunity, the deserializer code is very simple, it is less than 10 lines of code. If you get stuck I will be happy to help :)

So you need to do a String.replace to replace the characters not allowed in tiddler titles.

exports["application/x-bookmarks"] = function(text,fields) {

const parser = new DOMParser;

const vdoc = parser.parseFromString(text,"text/html");

const bookmarkAnchors = vdoc.querySelectorAll("a");

var results = [];

$tw.utils.each(bookmarkAnchors,function(anchor) {

results.push({

"add_date" : anchor.getAttribute("add_date"),

//"icon": anchor.getAttribute("icon"),

"title": anchor.textContent,

"url": anchor.getAttribute("href")

});

});

return results;

};

The other issue to consider is that in a bookmarks file, the title is not unique. So importing several bookmarks with the same title, only one will survive. So during the parsing, a unique title needs to be created.

[PMario]

On Friday, May 21, 2021 at 8:42:25 PM UTC+2 TiddlyTweeter wrote:

This is for both PMario & Mohammad ...

PMario wrote:

There is an API, to manage and manipulate bookmars with web-extensions.

So it should be possible to "manipulate" them from TW directly. ... But this would be a completely new AddON.

Ha! Interesting. What is the advantage of that API over BRUTE PARSING? 

Hi,

The html-export doesn't contain the bookmarks tags and description info. So information is lost. ... And I do like the idea to have a 2-way connection.

My browser bookmarks don't contain any tags, because the browser bookmark manager is very rudimentary. ..

Bookmarks can be synced among devices. So it would be nice to modify stuff on the PC and have them synced to the mobile. ...

Theoretically it should be possible to create browser addOns for all major browsers in 1 project. I would like to test this.

Just a view reasons ...

-mario

[PMario]

On Saturday, May 22, 2021 at 6:49:24 AM UTC+2 Mohammad wrote:

@TT, @PMario

Have a look at https://addons.mozilla.org/en-US/firefox/addon/tiddlymarker/

That's interesting. I'll have a closer look.

-m

[TiddlyTweeter]

Ciao Saq & Mohammad

saq.i...@gmail.com wrote:

Hi Mohammad, I think it is time for you to try your hand at some JavaScript :)

Right! I'd love to see the outcome of that. 

Likely Mohammad's new "LINK COMMANDER" TW-tool! :-)

 

This is a good opportunity, the deserializer code is very simple ...

... you need to do a String.replace to replace the characters not allowed in tiddler titles.

Right! It would be useful to see an example of String.replace in the code. Is that using REGULAR EXPRESSIONS? 

If so there I can help too. But I can't really read JS code. But I'm sure Mohammad will quickly learn how to :-)

Best wishes

TT

[Saq Imtiaz]

 

This is a good opportunity, the deserializer code is very simple ...

... you need to do a String.replace to replace the characters not allowed in tiddler titles.

Right! It would be useful to see an example of String.replace in the code. Is that using REGULAR EXPRESSIONS? 

If so there I can help too. But I can't really read JS code. But I'm sure Mohammad will quickly learn how to :-)

 @TT yes I believe Mohammad will be up and running quite easily, at least with the basics and that is where it all starts. That is how I got started with JavaScript as well many a year ago in the now defunct TW-Dev group.

String.replace can accept both a single string argument specifying what to replace, as well as a regular expression. 
Copying the examples below from the MDN link I posted earlier, note how the second example uses a regular expression to specify what to replace:

const p = 'The quick brown fox jumps over the lazy dog. If the dog reacted, was it really lazy?';

console.log(p.replace('dog', 'monkey'));

// expected output: "The quick brown fox jumps over the lazy monkey. If the dog reacted, was it really lazy?"

const regex = /Dog/i;

console.log(p.replace(regex, 'ferret'));

// expected output: "The quick brown fox jumps over the lazy ferret. If the dog reacted, was it really lazy?"

 

 

[Mohammad Rahmani]

On Sat, May 22, 2021 at 11:00 AM Saq Imtiaz <saq.i...@gmail.com> wrote:

Hi Mohammad,

Hi Saq,

I think it is time for you to try your hand at some JavaScript :)

Why not! let's give it a try ;-) :)

This is a good opportunity, the deserializer code is very simple, it is less than 10 lines of code. If you get stuck I will be happy to help :)

Sure!

So you need to do a String.replace to replace the characters not allowed in tiddler titles.

exports["application/x-bookmarks"] = function(text,fields) {

const parser = new DOMParser;

const vdoc = parser.parseFromString(text,"text/html");

const bookmarkAnchors = vdoc.querySelectorAll("a");

var results = [];

$tw.utils.each(bookmarkAnchors,function(anchor) {

results.push({

"add_date" : anchor.getAttribute("add_date"),

//"icon": anchor.getAttribute("icon"),

"title": anchor.textContent,

"url": anchor.getAttribute("href")

});

});

return results;

};

The other issue to consider is that in a bookmarks file, the title is not unique. So importing several bookmarks with the same title, only one will survive. So during the parsing, a unique title needs to be created.

Okay, we may add a number then! e.g title.... 1

On Saturday, May 22, 2021 at 7:19:37 AM UTC+2 Mohammad wrote:

Saq,

 I think the illegal letters by TW need to be scrapped e.g | [ ] { }

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/4e0e72be-405f-45b3-8e36-6d1a889abcafn%40googlegroups.com.

[Mohammad Rahmani]

Best wishes

Mohammad

;-) ;-)

 

Best wishes

TT

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/47f8ea7d-68ad-4fe8-945a-450a6180d046n%40googlegroups.com.

[TiddlyTweeter]

MULTIPLE LAUNCH?

So we busy COLLECTING links ...

But what about LAUNCHING them afterward? :-)

Surely, if you have a group of, say, 10 bookmarks, HOW will you launch, in a web session, all 10 at once?

My point is that to fully complement bookmark functionality in browsers one needs ALSO yo deal with MULTI-LAUNCH options :-)

Just a comment!

TT

[TiddlyTweeter]

TiddlyTweeter wrote... 

What is the advantage of that API over BRUTE PARSING? 

 

PMario replied ... (edited by TT)

The html-export doesn't contain the bookmarks tags and description info. So information is lost. ... 

Bookmarks can be synced among devices. So it would be nice to modify stuff on the PC and have them synced to the mobile. ...

 

Theoretically it should be possible to create browser addOns for all major browsers in 1 project. I would like to test this.

Right. I would certainly say that having TW be able to (through any method, external API, or whatever) handle linkage as smoothly as browser bookmarks is GOOD. Linkage, basically, is the gold of the net.

Naive comment

TT 

[Mohammad Rahmani]

Best wishes

Mohammad

On Sat, May 22, 2021 at 1:00 PM TiddlyTweeter <Tiddly...@assays.tv> wrote:

Ciao Saq & Mohammad

saq.i...@gmail.com wrote:

Hi Mohammad, I think it is time for you to try your hand at some JavaScript :)

Right! I'd love to see the outcome of that. 

Likely Mohammad's new "LINK COMMANDER" TW-tool! :-)

 

This is a good opportunity, the deserializer code is very simple ...

... you need to do a String.replace to replace the characters not allowed in tiddler titles.

Right! It would be useful to see an example of String.replace in the code. Is that using REGULAR EXPRESSIONS? 

@TT

What is the regex to replace the below chars with `-` at thesame time

|,  {,  }

 

I implemented the first part!

If so there I can help too. But I can't really read JS code. But I'm sure Mohammad will quickly learn how to :-)

Best wishes

TT

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/47f8ea7d-68ad-4fe8-945a-450a6180d046n%40googlegroups.com.

[TiddlyTweeter]

@TT

What is the regex to replace the below chars with `-` at thesame time

|,  {,  }

In raw JavaScript the regex match could be: /[|{}]/

The replace simply: /-/
IF you want to change "[" and "]" too try: /[|{}\[\]]/

I have NOT tested this so if there are issues let me know!

TT

[Mohammad Rahmani]

Do we need to capture what is in this between? 

consider a title like:   AIChE | American Institute of Chemical Engineers [2021] 

we need to remove [] and replace with -

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/4ccfc5e7-a333-4ad8-99c4-466e30f5f185n%40googlegroups.com.

[TiddlyTweeter]

Right.

Try the second match pattern I gave and let me know if it works for those cases ...  /[|{}\[\]]/

[Mohammad Rahmani]

@ TT

Check the attached revised one!

1. drag and drop the attached JSON in empath.html (download one from https://tiddlywiki.com/prerelease/)

2. save and reload

3. drag and bookmarks.html exported from FF/Edge/Chrome (I have one with nasty chars in the title)

4. See how the import works in Tiddlywiki

@Saq

Hoora! It works! Please see the code and advise!

1. I attached the revised deserializer and it works for titles with nasty characters.

2. Next step I will see how to use a unique title, help please

2.1. how can do this? should I use unusedtitle macro from TW core?

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/59ef0fc2-66b4-43d7-8469-ee0b97089bf7n%40googlegroups.com.

[Mohammad Rahmani]

@TT

Attached bookmark has nasty characters in the titles!

Look inside using a text editor! Then see how deserializer works here!

Sidenote:

I think this way we can also import phone books from android and ios ;-)

or we can import the history of today's web surfing in FF!

Saq provided a very powerful tool! His recently merged PRs allow users to drag and drop texts instead of files!

So, I think, many types of data can be imported, if a good deserialization is used! What do you think?

Best wishes

Mohammad

[Mohammad Rahmani]

@TT

Using the attached simple view template makes life easier!

I did extra tests with 850kB bookmarks no icons! And the Saq solution works like a charm!

I think it is good to create a created field, to have added bookmarks in the Recent tab!

But this is not good if one adds many bookmarks at once!

Best wishes

Mohammad

[Mohammad Rahmani]

More experiments:

Importing non ASCII but UNICODE title works like a charm!
See below image!

It is good if someone tests with bookmarks in other languages containing special chars!

[Saq Imtiaz]

Hi Mohammad, it looks good!

So we can't use the unusedtitle macro from TiddlyWiki because inside the deserializer code, these are not tiddlers yet.

What I recommend is to create another empty array where we make the array for results, called resultsTitles

Then inside the $tw.utils loop, where we have results.push, we should also for each bookmark do resultsTitles.push(title). Now we have an array of results titles.

So now we have to add an extra step for creating the title after the regex string replacement, here is pseudo code:

while title exists in resultsTitles array

   title = title + "1"

The title variable should now be unique for the bookmarks we are importing.

[Mohammad Rahmani]

Hi Saq,

On Sat, May 22, 2021 at 3:56 PM Saq Imtiaz <saq.i...@gmail.com> wrote:

Hi Mohammad, it looks good!

So we can't use the unusedtitle macro from TiddlyWiki because inside the deserializer code, these are not tiddlers yet.

What I recommend is to create another empty array where we make the array for results, called resultsTitles

Then inside the $tw.utils loop, where we have results.push, we should also for each bookmark do resultsTitles.push(title). Now we have an array of results titles.

So now we have to add an extra step for creating the title after the regex string replacement, here is pseudo code:

while title exists in resultsTitles array

   title = title + "1"

The title variable should now be unique for the bookmarks we are importing.

Okay. I got it! This checks to have unique titles in the array of bookmarks we import! I assume it does not check if there is any title in the wiki! fortunately the $:/Import interface shows us the duplicates!

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f73a724e-b8ad-4c81-9956-726b6341266an%40googlegroups.com.

[Mohammad Rahmani]

Hi Saq,

 This is the third revision! I used indexOf() and it seems it is compatible with ES5.

 I am not sure if the code is an efficient one or not, but it works!

To give it a try I have attached a bookmarks file containing characters on parsing to generate duplicates!

Best wishes

Mohammad

On Sat, May 22, 2021 at 3:56 PM Saq Imtiaz <saq.i...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f73a724e-b8ad-4c81-9956-726b6341266an%40googlegroups.com.

[Mohammad Rahmani]

Ah, there is an issue! It only searches for the first occurrence!

Needs to be corrected!

Best wishes

Mohammad

[TiddlyTweeter]

IF the regex on match is NOT GLOBAL flagged it will only match the first instance.

I am NOT familiar with JS regex flags but try this ...

/[|{}\[\]]/gi

Let me know!

TT

[Saq Imtiaz]

@Mohammad yes you need to have a while loop:

while(results.Title.indexOf(title) == -1) {

    title = title + "1";

}

Another improvement you can make just to have neater code, is to make a function called getTitle outside and before the $tw.utils.each loop.

var getTitle = function(title) {

    title = title.replace.....
     while(title in resultsTitles) {
        title = title + "1";

     }
    return title;

}

Inside this function you can do the string replacement of unwanted characters and also check for already existing titles in a loop, and return the new title. This is an aesthetic thing though and wont really impact the efficiency of the code.

@TT the regexp is good, Mohammad refers to check for duplicate titles only once instead of in a loop. So if "TiddlyWiki.com" already exists, we have to check if "TiddlyWiki.com1" already exists.

[Mohammad Rahmani]

Hi Saq,

 This is rev 4! I saw your comments later!

 This works and I use a JS Linter to check the code!

 I was not sure how to use var?! I made some changes with respect to R03.

 Now duplication is like Title 1, Title 2, ...

 Please have a look! I also added a nasty list of bookmarks for quick tests!

The regex I used the TT one with removing the last i, it does not make sense here! (ignore case)

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ffd4411a-a18f-419e-a348-b75c2d3460e1n%40googlegroups.com.

[Mohammad Rahmani]

On Sat, May 22, 2021 at 8:44 PM Saq Imtiaz <saq.i...@gmail.com> wrote:

@Mohammad yes you need to have a while loop:

while(results.Title.indexOf(title) == -1) {

    title = title + "1";

}

That is correct! I did a similar in R04! 

results.Title.indexOf(title)  

I did not know this!

Another improvement you can make just to have neater code, is to make a function called getTitle outside and before the $tw.utils.each loop.

var getTitle = function(title) {

    title = title.replace.....
     while(title in resultsTitles) {
        title = title + "1";

     }
    return title;

}

Very nice! So, one can define a new function in middle of the code!

 

Inside this function you can do the string replacement of unwanted characters and also check for already existing titles in a loop, and return the new title. This is an aesthetic thing though and wont really impact the efficiency of the code.

Perfect!

@TT the regexp is good, Mohammad refers to check for duplicate titles only once instead of in a loop. So if "TiddlyWiki.com" already exists, we have to check if "TiddlyWiki.com1" already exists.

Thank you for all your help! I would appreciate sharing your final version! and give a link for the community!

I think that my version is not good enough to be used by other users!

(p.s I am not sure if these deserializers may be part of core later or not! If not, it is worth having them as a plugin!)

I will write small help for potential users!

 

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ffd4411a-a18f-419e-a348-b75c2d3460e1n%40googlegroups.com.

[Saq Imtiaz]

Hi Mohammad! As per our email discourse, your code is good. Polish it up and publish the first kookma JavaScript plugin!

[Mohammad Rahmani]

Hi Saq,

 Many thanks for all your help! Sure I will! Of course this is a product from TW Colab! ;-) :-)

I add documentation and soon I will push it to GitHub!

I also appreciate all the help from Josih!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d7ff28ea-d806-4f79-9475-4ee7a87d3dd6n%40googlegroups.com.

[TiddlyTweeter]

This for both Mohammad & Saq

Mohammad wrote:

I add documentation and soon I will push it to GitHub!

Good!

I have been testing it and DISCOVERED is NOT ONLY for Bookmark exports.

It will process ANY saved webpage. Where you append ".bookmarks" to it's name.

In other words it is a GENERIC MINI-PARSER for properly formed <a> links.

I did tests with saved WikiPedia pages. It  worked perfectly ...

My POINT is this is NOT JUST a tool for parsing Browser Bookmarks. IT will parse anything!

I seriously think that needs to be CLEAR in docs.

Best wìshes

TT

[Saq Imtiaz]

@TT very true. I took a shortcut with parsing the bookmarks file format and directly grabbed the <a> elements since those are the only elements with data of interest to us.

Note that the .bookmarks extension is just a way to tell TW to use this deserializer to parse this file. The code can easily be tweaked to use a different extension instead that might be more intuitive.

[Mohammad Rahmani]

Hi TT,

 Interesting! That means one can extract all links in a webpage! For example I am visiting aiche.org and if save and add .bookmarks to file name then drag and drop into bookmarker!

 Tiddlywiki will extract all links!

That is a MAGIC ;-) 

Best wishes

Mohammad

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/fcea22cd-12c9-4e35-a1cb-fe5cb6090c5cn%40googlegroups.com.

[TW Tones]

Folks,

In a related matter, scraping content from a site. I started recently to collect CSS attributes and variables into a TW, when wanting to Align something I thought a search for align will allow be to quickly see the options and variations eg align image vs text. The aforementioned method strip links from html would help, primarily providing links back to more exhaustive references online. But in this endeavour I still need to find a better data source I can convert into tiddlers And establish the relationships with the CSS syntax context and values that can be used.

I mention this because this kind of work stands to make helpful wiki editions that may be distributed more widely than regular custom and personal wikis. Add to such a "database" of CSS analytics that monitor the users lookups and copy to clipboard activity, then provide learning support such as flashcards and possibly even fuzzy logic to support learning and we would have a winner (or many).

Screen scraping and content organising is highly achievable in tiddlywiki, we handle csv, text, json and csv quite well (could be better) but these typically need bespoke methods. Perhaps in time we can make this easier and empower more people to build there own solutions by incorporating "found data".

I myself plan to launch a wiki containing a manual for a modem commonly used in Australia for which there is not real manual for, and considering a tool for individuals or businesses to plan how they address the  Sustainable Development Goals. Freely given but with my branding attached.

Regards

Tones

[TiddlyTweeter]

Ciao Mohammad & Saq ...

Mohammad wrote:

That means one can extract all links in a webpage! For example I am visiting aiche.org and if save and add .bookmarks to file name then drag and drop into bookmarker!

 Tiddlywiki will extract all links!

That is a MAGIC ;-) 

Right!

An issue with The Power of GENERIC <a> EXTRACTION ...

1 - You could end up with so MANY Tiddlers it becomes a nightmare to manage! :-) 

   so ...

2 - I SUGGEST each ".bookmarks" import need a UNIQUE ID adding (tag or other field unique IDENTIFIER) so you can DELETE them all if needed (JUST that specific import).

   WHY? 

•     In tests with extracting from WikiPedia pages I found out that you can easily add over 200 "Link Tiddlers" to a TW. 

•     In short, with Scale potential one also needs Scale Removal options too! :-) so ...

3 - PROVIDE USER with a macro to selectively DELETE Tiddlers created by (2).

Just thoughts!

TT

[TiddlyTweeter]

Mohammad wrote:

 Interesting! That means one can extract all links in a webpage! 

Right and MORE.

For example it could be a way to import an entire EMAIL ADDRESS BOOK into TW.

All that is needed is the input file contains <a> links of them.

There is a LOT of potential in the approach I think!

Best wishes

TT

[Mohammad Rahmani]

On Mon, May 24, 2021 at 12:25 PM TiddlyTweeter <Tiddly...@assays.tv> wrote:

Ciao Mohammad & Saq ...

Mohammad wrote:

That means one can extract all links in a webpage! For example I am visiting aiche.org and if save and add .bookmarks to file name then drag and drop into bookmarker!

 Tiddlywiki will extract all links!

That is a MAGIC ;-) 

Right!

An issue with The Power of GENERIC <a> EXTRACTION ...

1 - You could end up with so MANY Tiddlers it becomes a nightmare to manage! :-) 

Yes dragging a webpage is not a good idea!

   so ...

2 - I SUGGEST each ".bookmarks" import need a UNIQUE ID adding (tag or other field unique IDENTIFIER) so you can DELETE them all if needed (JUST that specific import).

   WHY? 

•     In tests with extracting from WikiPedia pages I found out that you can easily add over 200 "Link Tiddlers" to a TW. 

•     In short, with Scale potential one also needs Scale Removal options too! :-) so ...

3 - PROVIDE USER with a macro to selectively DELETE Tiddlers created by (2).

Right! I use Tiddler Commander the latest!

After import, TW shows me a button for further processing I can tag them! Add extra fields:values or delete a bunch of them!

 

Just thoughts!

TT

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d8d21c80-0a61-4167-b8df-66fadf65a7c5n%40googlegroups.com.

[TW Tones]

Folks,

Doing anything in bulk capable of generating a lot of tiddlers would be supported by using the method the JSON Mangler uses to generate tiddlers from CSV files. pack them into a plugin. Shadow tiddlers are easy to access and can be overridden with edits as needed. Non-plugin JSON tiddlers (of tiddlers) are less easy to access without additional tools. We also have zipping at our disposal but a custom JSON structure to store bookmarks can be made more efficient but less reusable (without additional tools)

Tones  

[Mohammad Rahmani]

On Tue, May 25, 2021 at 3:51 AM TW Tones <anthony...@gmail.com> wrote:

Folks,

Doing anything in bulk capable of generating a lot of tiddlers would be supported by using the method the JSON Mangler uses to generate tiddlers from CSV files. pack them into a plugin. Shadow tiddlers are easy to access and can be overridden with edits as needed. Non-plugin JSON tiddlers (of tiddlers) are less easy to access without additional tools. We also have zipping at our disposal but a custom JSON structure to store bookmarks can be made more efficient but less reusable (without additional tools)

Hi Tones! I have not tested JSON Mangler for importing bookmarks, but there are some addons for FF and Chrome that lets you export History and Bookmarks to a JSON file! These files then can be imported to Tiddlywiki! The cost is installing such addons!

 

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/05c69b56-0ad4-4e10-bd43-49fc556cb472n%40googlegroups.com.

[TW Tones]

Mohammad,

The Key to my prior post is intended to be the Generated tiddlers become shadows within a plugin. Just as deleted tiddlers are in the trashbin (i Think).

Tones

[dieg...@gmail.com]

Hey Mohammad.

Was this ever pushed to github?

[Mohammad Rahmani]

Hi Diego,

 I totally forgot about this! I will upload to GitHub this weekend!

The JS code has been finalized by Saq!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9496f2f3-9605-4e76-8178-fecae17ca376n%40googlegroups.com.

[TW Tones]

Folks,

Do we have a final product as a result of this thread?

Could a link to this published work be share in a post at the bottom of this thread please. 

Thanks for the effort.

Tones

[Mohammad Rahmani]

Hi Tones

Actually I was working on a bookmarker plugin implementing different tools! Saq has finalized the JS code and has given me very useful inputs.

I promised Diego to publish it! But I am very busy and I hope I can do this in my free time!

By the way, I can post the JS tiddler (deserializer) if you can use it directly!

Sorry @Diego Mesa for not finishing this!

I really miss the TW community, writing TW scripts, and developing new plugins. I hope I can return not too late!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2ad0f631-df81-4659-96b1-02e34c6978fcn%40googlegroups.com.

[TW Tones]

Mohammad,

All in good time, I just did not want to miss out. I extended your simple basket plugin because its great, perhaps a bookmark basket?

"We Always miss you" too (he says genuinely) but life must come first.

Regards

Tones

[Mohammad Rahmani]

Hi Tones,

 Thank you! This is the latest deserializer developed and approved by Saq.

See the details in this thread!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/20af900e-6538-4017-af93-495d2cb110b9n%40googlegroups.com.