NoteSelf the Evernote-like experience update

[Danielo Rodríguez]

Hello Early adopters!

I just released a new version of noteself (https://noteself.github.io/)

This is an important internal update. All the tiddler configurations are now stored on a local database. This may sound as a small thing but it is an important improvement.

Thanks to this feature it is now possible to use NoteSelf online, without downloading or installing anything. So anyone that wants to try it out without even downloading the file it is now possible.

I have also updated the authentication mechanism and it fails much less now.

So, please try it out at:

https://noteself.github.io/

[Matabele]

Hi Danielo

Great job! 

regards

[Thomas Elmiger]

Looks great, bookmarked for later!
Regards, Thomas

[Danielo Rodríguez]

I just released a small patch to make easier databases switching.

Hope no one tried to use it before, because it was a bit confusing. Not it works much better.

Regards

[Mat]

I have at a distance followed your quest on these matters over the past few years. Really cool that it is actually real now. But - apropos my own adventures with TWederation - it is funny how other peoples projects can just be so much more difficult to understand. I struggle understand benefit with this browser db over just having a TW hosted just anywhere? Is this it:

When using NoteSelf locally you don't have to save, because this is done automatically. If wanting to save in cloud, you must still click save.

Normal local TW doesn't save automatically so one must click save both for local and for cloud save.

This local autosave is of course good - but am I missing something? I mean, its not THAT difficult to click save every now and then, so is there more to it? 

Admittedly my Win10 crashes even more often than FF these days so auto-stored work would really be good on those occasions.

When contrasting NoteSelf to Evernote I can see tremendous advantages with NoteSelf, but those are more from native TW than NoteSelf - or? 

Now, that said, I notice the FAQ question: "How can I sync multiple devices?" - now that would be really cool, i.e if one didn't have to upload anything but NoteSelf was "riding on" the automatically synching that browsers do if one is logged into ones own browser account. That would make a kind of tiddly-intra-net between all ones own devices and that autosynchs whenever one is online. Very useful!

<:-)

[Danielo Rodríguez]

Hello Mat, first of all, thank you for your valuable feedback. You have a very good eye for detail and usually spot ideas and issues that no one else does, so thanks in advance.

El domingo, 21 de agosto de 2016, 23:54:40 (UTC+2), Mat escribió:

it is funny how other peoples projects can just be so much more difficult to understand.

Things are usually clear for the ones who envision them, but it is hard to translate that idea.

 

When using NoteSelf locally you don't have to save, because this is done automatically. If wanting to save in cloud, you must still click save.

Normal local TW doesn't save automatically so one must click save both for local and for cloud save.

Reading again your sentence to be able to answer you accordingly I just realized about the source of your confusion (I think). The point is to not store anything on the TW file, nothing, zero, and save everything on the  local pouch database, the sync adaptor takes care of that. If you have a cloud configured for the currently selected database (you can have several databases) you just have to login. Once you are logged in ,cloud sync will happen automatically in background. A possible analogy could be a node wiki that saves tiddlers to a dropbox folder and dropbox takes care of syncing those files to dropbox servers and across devices.

 

This local autosave is of course good - but am I missing something? I mean, its not THAT difficult to click save every now and then, so is there more to it? 

Admittedly my Win10 crashes even more often than FF these days so auto-stored work would really be good on those occasions.

It is not only about autosave. It's about synchronization, having several databases in one single TW, is about not having to download anything OR downloading it if you want. Basically all what TW is but adding synchronization to the mix. 

The MAIN difference between storing a wiki online in tiddlyspot and using NoteSelf is that you don't need to be online to use NoteSelf. You can download NoteSelf to your computer, configure a remote server and work totally offline. When you back online, all your data is synced to the remote database. Then, in another computer you can just use the online version of NoteSelf, configure the same remote server and get the same data synced to that instance. 

Currently I use NoteSelf on my work, and on my home I also use it but I have two databases configured (in the same file) one that syncs with my work DB and another that Syncs to my personal one, without mixing any data or having several files.

NoteSelf arises from my need of being able to take small notes on my mobile or tablet and have them available on my computer and the other way round. Using dropbox for this task is not a good option because:

• Syncing an entire file of 5MB each time I take a small note is totally overkilling. Specially on my phone and my limited data plan.
• When I take a note on my tablet it can take some time until I connect it to a wifi network. On that time I may forget that I did a change on my tablet, make a change on my computer and -BOM- you have a file conflict on the best case, and loss of information in the worst.
• Keeping a wiki in sync across several devices, even using dropbox, could be a nightmare.  

 

When contrasting NoteSelf to Evernote I can see tremendous advantages with NoteSelf, but those are more from native TW than NoteSelf - or? 

As it's said on the page, I took the best of TW and added sync to the mix, so yes, most of the advantages comes from TW. But, the idea is to provide not just a TW edition, but a small ecosystem. I have plans to build an android application, a chrome app and maybe an ad-hoc server for making this easier for novice users.

Some of other advantages that are on the roadmap are:

• Synchronization of plugins. This means that each database (you can understand it as NoteBooks) can have it's own set of plugins that will be synced across your devices. And switching between databases means also loading different plugins and customizations.
• Tiddlers revisions. This is happening right now, but there is no interface to easily switch between revisions

 

Now, that said, I notice the FAQ question: "How can I sync multiple devices?" - now that would be really cool, i.e if one didn't have to upload

I just realized that I didn't included this feature in the features list, which is a tremendous error, specially because it is the MAIN focus of the entire thing.

 

anything but NoteSelf was "riding on" the automatically synching that browsers do if one is logged into ones own browser account. That would make a kind of tiddly-intra-net between all ones own devices and that autosynchs whenever one is online. Very useful!

That is indeed a good idea Mat, but I think that is extensions territory. I have been investigating about it and (as usual) there are lots of restrictions imposed by the browsers, and the APIs are only accessible to extensions. 

Thank you very Much Mat, and hope you give NoteSelf a try, your feedback would be very valuable.

[Josiah]

Ciao Danielo & Mat

I am really fascinated by this & will try it out in detail.

WHY? Perhaps not for reasons others might.

I find it really interesting as a possible way to get away from all the browser & platform plugins.

As far as i understand in Danielo has developed a way of saving tiddlers that is NEITHER directly dependent on servers NOR on browsers add ons.

Is that so? If so it approaches & facilitates a uniform user experience across platforms and offline/online.

At the very least its indicative a UNIVERSAL "Out-of-the-box" TW may well work & can get round browser limits. Why have loads of variant ways when ONE will do?

Best wishes
Josiah

[Brian Theado]

Hi Danielo. Thanks for sharing your plugin, it looks really interesting.

On Mon, Aug 22, 2016 at 11:13 AM, Danielo Rodríguez <rdan...@gmail.com> wrote:

[...] 

It is not only about autosave. It's about synchronization, having several databases in one single TW, is about not having to download anything OR downloading it if you want. Basically all what TW is but adding synchronization to the mix. 

[...] 

I can see it isn't only about autosave (nice explanation), but the autosave without using a browser plugin is a really nice feature on its own (as Josiah also mentioned). If I were to use this plugin for the autosave functionality only (i.e. without a couchdb server), I would feel a little nervous about have all my tiddler data stored only in the browser's IndexedDB storage. Rather than being diligent about backing up my browser's profile directory, I got to thinking it would be nice to have a download button which downloads a full tiddlywiki including all the tiddlers from the pouchdb database. That way I could autosave to browser storage throughout the day and maybe once or twice a day hit the download button to store a full tiddlywiki file as the backup.

I was even thinking that when the user does not have a couchdb server configured, the default behavior for download could be to download the full tiddlywiki. Because it is only when the couchdb server is configured that the "thin" tiddlywiki is useful. What do you think?

But then that brings me to a further idea that the plugin could notice when a "thick" version of a TW file is loaded and it could automatically sync the tiddlers into the pouchdb localstore. Seems like that might be tricky to get right, but the idea is that the "thick" tiddlywiki file becomes a replacement for a couchdb server. You could send it through email, copy via usb, save via dropbox (i.e. maybe a few times a day to save your data plan), etc. and resync using that file at the destination browser. You get the sync feature of a couchdb server without ever having to be online.

This "import into pouchdb" functionality would also be useful if someone had an existing tiddlywiki and they wanted to start using the pouchdb plugin with it.

I know these ideas are not in line with your main use case, but I just wanted to share.

Thanks,

Brian

[Mat]

Danielo, thanks for reply. Still need clarification though:

So synching does not treat the full TW as one unit but instead only synchs the changes, right? That seems very good and should significantly speed up things compared to up/downloading full TW files.

It is not only about autosave. It's about synchronization, having several databases in one single TW,

Which "several databases" are those? Is it one db per device + the cloud stored one? Is the content of each database a subset of tiddlers? Or is it the full set of common tiddlers plus the specific ones that one happens to edit at that device? Maybe the data you refer to is on another level - partial tiddlers in bits and bytes synching, for example changing a single tag only transfers that snip of code? Sorry for the many questions.

The MAIN difference between storing a wiki online in tiddlyspot and using NoteSelf is that you don't need to be online to use NoteSelf. You can download NoteSelf to your computer, configure a remote server and work totally offline. When you back online, all your data is synced to the remote database. Then, in another computer you can just use the online version of NoteSelf, configure the same remote server and get the same data synced to that instance. 

But... you don't' have to be online to use a [copy of] what is stored on tiddlyspot either (!?). It is loaded on your local computer like any webpage, you modify it offline or online and whenever you're online you can upload it. The only difference seems to be that if I want to save my changes while still offline then when I click save it is stored as a local file.

 

The conflict with loss of data seems to be a problem in both variants.

I have more questions but I must test some things first.

Thank you very Much Mat, and hope you give NoteSelf a try, your feedback would be very valuable.

Yes, even if my main mission right now is TWederation. Who knows, maybe NoteSelf is somehow applicable there? Any ideas?

I'll finish off with a gift:

Error report:

Earlier today, or maybe yesterday, I used the demo and created one tiddler and a second one I left in edit mode to see how it would be presented after reload. It was presented as a saved draft, i.e the title was "Draft of...". Now thhat I once again to to the noteSelf site and open the demo, they appear again but when I click edit on the "Draft of.." tiddler, I get:

syncer-browser - 22:15:11 22 8 2016

Sync error while processing 'Draft of 'Draft of 'New Tiddler': {"status":409,"name":"conflict","message":"Document update conflict"}

<:-)

[Mat]

On Monday, August 22, 2016 at 6:00:52 PM UTC+2, Josiah wrote:

[...]a UNIVERSAL "Out-of-the-box" TW may well work & can get round browser limits. Why have loads of variant ways when ONE will do?

Interesting and valuable observation!!! I have no competence to tell if it would work, but thats intriguing.

I find it really interesting as a possible way to get away from all the browser & platform plugins. 

Just so I understand fully; you're (only) referring to tiddlyfox and tiddlychrome, right? Or are there more plugin dependencies?

<:-)

[Brian Theado]

Replying to my own message as I was mistaken. See below

On Mon, Aug 22, 2016 at 1:49 PM, Brian Theado <brian....@gmail.com> wrote:

I can see it isn't only about autosave (nice explanation), but the autosave without using a browser plugin is a really nice feature on its own (as Josiah also mentioned). If I were to use this plugin for the autosave functionality only (i.e. without a couchdb server), I would feel a little nervous about have all my tiddler data stored only in the browser's IndexedDB storage. Rather than being diligent about backing up my browser's profile directory, I got to thinking it would be nice to have a download button which downloads a full tiddlywiki including all the tiddlers from the pouchdb database. That way I could autosave to browser storage throughout the day and maybe once or twice a day hit the download button to store a full tiddlywiki file as the backup.

I was even thinking that when the user does not have a couchdb server configured, the default behavior for download could be to download the full tiddlywiki. Because it is only when the couchdb server is configured that the "thin" tiddlywiki is useful. What do you think?

I didn't test well enough and I fooled myself into thinking the main TW download/save button in the sidebar was not saving the tiddlers stored in the browser storage. But when I looked closer, I found the pouchdb tiddlers were there in the downloaded html file.

So the download/save button can already be used as a method for backing up the pouchdb data.

[Danielo Rodríguez]

El lunes, 22 de agosto de 2016, 18:00:52 (UTC+2), Josiah escribió:

Ciao Danielo & Mat

Hello Josiah

I am really fascinated by this & will try it out in detail.

Cool! please share back what you find odd, confusing or what you miss. I really need users input to improve the UI.

 

 

As far as i understand in Danielo has developed a way of saving tiddlers that is NEITHER directly dependent on servers NOR on browsers add ons.

I'm not sure that I understand this fully. My solution does not depends DIRECTLY of any of those things, but it requires that the browser supports local storage (which 99% of them do). It does not depends directly from a server because it is optional, but if you want to sync between several devices you need a server (at least for now).

 

Is that so? If so it approaches & facilitates a uniform user experience across platforms and offline/online.

That's exactly my target. An uniform experience.

 

At the very least its indicative a UNIVERSAL "Out-of-the-box" TW may well work & can get round browser limits. Why have loads of variant ways when ONE will do?

Well, currently it suffers from browsers limits because it is not able to do some things the browsers restrict, but I want to get rid of those restriction building platform specific versions. The first one would be the Android app.

Thanks for giving it a try. 

[Danielo Rodríguez]

El lunes, 22 de agosto de 2016, 19:50:03 (UTC+2), Brian Theado escribió:

Hi Danielo. Thanks for sharing your plugin, it looks really interesting.

Hello Brian. Thank you for giving it a try!!

 

I can see it isn't only about autosave (nice explanation), but the autosave without using a browser plugin is a really nice feature on its own (as Josiah also mentioned). If I were to use this plugin for the autosave functionality only (i.e. without a couchdb server), I would feel a little nervous about have all my tiddler data stored only in the browser's IndexedDB storage. Rather than being diligent about backing up my browser's profile directory, I got to thinking it would be nice to have a download button which downloads a full tiddlywiki including all the tiddlers from the pouchdb database. That way I could autosave to browser storage throughout the day and maybe once or twice a day hit the download button to store a full tiddlywiki file as the backup.

I perfectly understand how do you feel, because it's something I think about it several times. Indeed when I wrote the first version of the plugin (wow, more than a year ago!) that was one of my first targets and goals. But I started to leave it aside and I finally forgot about it on the first release. Why? Because I was unable to anticipate user's preferences, in fact I was unable to anticipate my own preferences. I mean, what would be better? To download a JSON file representing the database? To download a copy of the wiki with all the tiddlers ? Should such version include the plugin or should it get rid of it? Should the downloaded version be composed of regular plain tiddlers or just have one single tiddler representing the whole database? Should the exported versions of the database include all the tiddler's revisions or just the last one? As you can see, a lot of questions I'm not able to answer, not to mention that they were difficult to implement for me at that moment. Now that I'm starting to get some user feedback I may take better decisions.

 

I was even thinking that when the user does not have a couchdb server configured, the default behavior for download could be to download the full tiddlywiki. Because it is only when the couchdb server is configured that the "thin" tiddlywiki is useful. What do you think?

As I have mention, that was indeed the intention, but was harder than I expected. Tiddliwiki does not provide a method to generate what should be downloaded, it's TW core who generates the content and then passes it to the saver, so I would need some kind of hook for such feature, but it is definetively on the roadmap.

 

But then that brings me to a further idea that the plugin could notice when a "thick" version of a TW file is loaded and it could automatically sync the tiddlers into the pouchdb localstore. Seems like that might be tricky to get right, but the idea is that the "thick" tiddlywiki file becomes a replacement for a couchdb server.

What do you mean with a "thick" version? A regular TW which contains tiddlers that are not saved on the local storage? That could only happen if you install the plugin in an already existing database. I don't have plans to support that configuration as a priority, because my priority is that NoteSelf is used as a TW edition. Anyway, if you could expand on your definition of "thick" version I will be very grateful because seems like an interesting idea.

 

This "import into pouchdb" functionality would also be useful if someone had an existing tiddlywiki and they wanted to start using the pouchdb plugin with it.

Now that I'm thinking about it it does not looks so difficult... An interactive version would be definitely easier. An automatic one could be a bit pain in the neck.

 

I know these ideas are not in line with your main use case, but I just wanted to share.

And they are Very welcome! Please don't hesitate to share all the ideas you get.

Regards 

[Josiah]

Ciao Mat & Danielo

J. >> [...]a UNIVERSAL "Out-of-the-box" TW may well work & can get round browser limits. Why have loads of variant ways when ONE will do?

M. > Interesting and valuable observation!!! I have no competence to tell if it would work, but thats intriguing.

I'm not competent either. But I'm gonna give it a whirl in a bunch of browsers.

What stands out, at the very least, is a PROOF OF CONCEPT.

As far as I understand the history of TW the more restrictive limits on file saving BROWSERS have enforced, and in different ways, have made its life much more complicated than has been healthy for it. In that sense its been a bit of a victim of others' limitations. The "cruft" needed to get round these limits makes it not so easy to enter for many first-timers IMO.

J. >> I find it really interesting as a possible way to get away from all the browser & platform plugins.

M. > Just so I understand fully; you're (only) referring to tiddlyfox and tiddlychrome, right? Or are there more plugin dependencies?

IE has one of sorts. But actually I was not that clear there. I meant any and all browser plugins, as well as work-arounds. And dealing with currently UN-supported browsers.

Anyway to get away from these exceptions to a simpler way to save gets my attention.

It seems to me that BROWSERS enforced (for security reasons?) limits that have really made life difficult on TW file saving, backing-up etc. The use of a local database seems to be a way they have uniform(ish)  standards on as an alternative?

I don't yet know enough. But I do think its well worth looking at and thinking about. Danielo's TW instances how it can be done.

Best Wishes
Josiah

[Josiah]

Ciao Danielo

J. >> I am really fascinated by this & will try it out in detail.

D. > Cool! please share back what you find odd, confusing or what you miss. I really need users input to improve the UI.

I will. My intention is to try it across several browsers. It may take me some weeks to report back.

J. >> As far as i understand in Danielo has developed a way of saving tiddlers that is NEITHER directly dependent on servers NOR on browsers add ons.

D. > I'm not sure that I understand this fully. My solution does not depends DIRECTLY of any of those things, but it requires that the browser supports local storage (which 99% of them do). It does not depends directly from a server because it is optional, but if you want to sync between several devices you need a server (at least for now).

Yes. I meant that. Servers are involved in syncing, but its AGNOSTIC. I mean its not totally dependent on one specific server in order to function. The servers are intermediaries, not bosses. If you get what I mean?

J. >> Is that so? If so it approaches & facilitates a uniform user experience across platforms and offline/online.

D. > That's exactly my target. An uniform experience.

That's what it looks like its got very close to. The approach you have developed I believe highlights & potentially solves many issues in TW being able to work consistently the same across platforms & browsers on saving & sync. I understand there are still some differences in play between browsers, but they seem far less problematic than when using standard TW.

Best wishes
Josiah

[t...@camandtom.com]

Hey Danielo,

Great effort! Would love to give this a try, but struggle to get this set up. I've created a cloudant couchdb instance at https://ohc.cloudant.com/. Trying to configure this in my local NS page, I get a pop-up saying XMLHTTPRequest error code:0. The javascript console displays the following error:

XMLHttpRequest cannot load https://ohc.cloudant.com/_session. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.

$:/core/modules/utils/logger.js:30 syncer-browser: XMLHttpRequest error code: 0

What am I doing wrong?

Thanks,

Tom

[Danielo Rodríguez]

El martes, 23 de agosto de 2016, 4:13:47 (UTC+2), Brian Theado escribió:

Replying to my own message as I was mistaken. See below

Hello again Brian

I didn't test well enough and I fooled myself into thinking the main TW download/save button in the sidebar was not saving the tiddlers stored in the browser storage. But when I looked closer, I found the pouchdb tiddlers were there in the downloaded html file.

So the download/save button can already be used as a method for backing up the pouchdb data.

Let me be 100% clear on this. The download button at this moment behaves exactly as it normally does: downloads a copy of the currently loaded wiki. Please note the emphasis at loaded wiki. This small detail has one big and important fact: Only the data that is currently loaded in memory is included. What does this mean? By default, when tiddlywiki uses a sync-adaptor it first requests to it all skinny tiddlers. Those are tiddlers without the text field. When that tiddler is opened then the text field is downloaded. Let's review one example:

1. You open your NoteSelf wiki. All the skinny tiddlers are loaded from the local DB. Those tiddlers are not completed, they miss the text field.
2. You add some tiddler with some text. That tiddler is saved to the local database and to the wiki store (currently loaded wiki)
3. You save the wiki. What you will get is a TW with your tiddlers for this editing sessions and maybe those that you have opened for reading.

To be honest, I can easily fix this situation by returning the FAT(full) tiddlers when skinny ones are requested, but the convention is to not do that. Also it could important impact on memory usage and performance because all the tiddlers would be loaded into memory. This may not be very important on desktops, but it is on mobiles and slow computers where TW behaves very slowly. 

Maybe I can include it as a configurable option and let every user chose what they want.

[Danielo Rodríguez]

Hello Tom,

Sorry for this situation. I want to include a 2-min setup guide video. And I also want to include this on the FAQ section, so thank you for the reminder.

Fortunately this has an easy solution for you :D. You have to enable CORS on your cloudant account. To do so follow this steps:

1. Log in into your cloudant account
2. Click on the account section
3. Once on the account section click on CORS
4. At your right you should see a button "Enable CORS". Click that button
5. I suggest you to allow all domains. If you want, you can restrict to noteself.github.io, but that is not necesary

That's all!! See the following screenshot for a visual guidance:

[Danielo Rodríguez]

El martes, 23 de agosto de 2016, 13:06:55 (UTC+2), Josiah escribió:

Ciao Danielo

Hello  Josiah

I will. My intention is to try it across several browsers. It may take me some weeks to report back.

That is very cool, thanks. Please note that during that period I may release some patches.

 

 

Yes. I meant that. Servers are involved in syncing, but its AGNOSTIC. I mean its not totally dependent on one specific server in order to function. The servers are intermediaries, not bosses. If you get what I mean?

Yes I think that I understand it now.

 

The approach you have developed I believe highlights & potentially solves many issues in TW being able to work consistently the same across platforms & browsers on saving & sync. I understand there are still some differences in play between browsers, but they seem far less problematic than when using standard TW.

Ok, this clarifies totally what you meant. Normally, on each platform TW saving mechanisms are very different: download save, plugin save (tiddyfox, tiddyChrome), using Android app (a bit outdated), using IOS app, and all of them behave different

Same for server syncing: upload to a php server, syncing to a remote nodejs using the filesystem plugin, syncing to a different server using a different plugin... and all of them are incompatible with each other. NoteSelf in the other hand behaves the same for all of those situations: Saving locally uses the same workflow that syncing to a remote server, no difference, and both saving and syncing methods are exactly the same on mobile and desktop versions. Good point of view, thanks!

[Danielo Rodríguez]

El lunes, 22 de agosto de 2016, 22:24:28 (UTC+2), Mat escribió:

Danielo, thanks for reply. Still need clarification though:

No problem, as much as you need.

 

So synching does not treat the full TW as one unit but instead only synchs the changes, right?

Correct.

 

Which "several databases" are those? Is it one db per device + the cloud stored one?

Thank you for pointing that out. I feel this needs clarification. It's not only one db per device, you can do it that way, but you are not restricted to it.

 

Is the content of each database a subset of tiddlers?

Exactly. Each database is just a subset of tiddlers, not an entire TW. You choose if you want everything in one bundle or if you want to split. But because tiddlers are also parts of TW each database can result on a TW with a totally different look, everything on the same file depending on which database you choose.

 

Or is it the full set of common tiddlers plus the specific ones that one happens to edit at that device? Maybe the data you refer to is on another level - partial tiddlers in bits and bytes synching, for example changing a single tag only transfers that snip of code? Sorry for the many questions.

No, nothing so complicated. Just what I said above. Think in databases as they were just notebooks.

 

But... you don't' have to be online to use a [copy of] what is stored on tiddlyspot either (!?). It is loaded on your local computer like any webpage, you modify it offline or online and whenever you're online you can upload it.

Yes, a copy of it, an static copy. If you made some changes (new tiddlers), and someone add new tiddlers to the online one, then when you save your offline copy someone is going to loose data. Note self keeps a copy of all the tiddlers, that could look the same, and then when you back online the databases "communicates" between them and exchange/share tiddlers. You don't need to take care anymore of reviewing that your save does not overwrites some changes that are not present on the copy you are saving.

 

The only difference seems to be that if I want to save my changes while still offline then when I click save it is stored as a local file.

Apart from what I have explained, this is another important difference. If you click save, it is stored as a local file, true, and then you have to open that file to be able to continue editing and adding more content. If you open a previous file by mistake (and this have happened to me several times) you will find yourself again in the situation of losing data.

 

 

The conflict with loss of data seems to be a problem in both variants.

Could you explain why? Of course nothing is rock solid, but I find NoteSelf to be very consistent.

 

I have more questions but I must test some things first.

Have fun!

 

Yes, even if my main mission right now is TWederation. Who knows, maybe NoteSelf is somehow applicable there? Any ideas?

I hope so, but I need a deeper knowledge on TWederation to provide ideas.

 

I'll finish off with a gift:

Lovely! 

 

Error report:

Earlier today, or maybe yesterday, I used the demo and created one tiddler and a second one I left in edit mode to see how it would be presented after reload. It was presented as a saved draft, i.e the title was "Draft of...". Now thhat I once again to to the noteSelf site and open the demo, they appear again but when I click edit on the "Draft of.." tiddler, I get:

How have you ended with a tiddler entitled "Draft of Draft of New tiddler" ? WOW, I just tried what you described and I ended with a "Draft of 'Draft of 'Draft of 'New Tiddler 1'''" tiddler. In any case, you can safely ignore the name conflicts for drafts as long as you give a name to the tiddler and save it without any error.

[Mark S.]

As an Evernote replacement, isn't there a problem with images? That is, unless you have the images you want to embed on a 2nd service, there's no way to make them portable with this process.

Thanks!
Mark

[Danielo Rodríguez]

Hello Mark S.

Tiddlywiky, and by extension Noteself, supports images. I tried with some small images (50k) and I can embed them with no problems.

Worth mentioning that the database that I'm using supports attachments, which is a more efficient way of including binaries than using base64. It's not implemented yet but is on my list.

What problem do you see exactly?

[Mark S.]

Images can fill up TW pretty fast, slowing it down to a crawl. If you want to include higher-resolution images (think maps) then you really need to park them outside TW. My external image collection is just about 50megs on my main TW.

How do attachments work? I'm guessing they use binary as storage rather than the puffy base64. That's an improvement, but there's nothing that will make 1meg jpeg take up less than 1mb without loss of quality or resolution.

Thanks!
Mark

[Brian Theado]

Danielo,

On Tue, Aug 23, 2016 at 11:03 AM, Danielo Rodríguez <rdan...@gmail.com> wrote:

[...] 

Only the data that is currently loaded in memory is included. What does this mean? By default, when tiddlywiki uses a sync-adaptor it first requests to it all skinny tiddlers. Those are tiddlers without the text field. When that tiddler is opened then the text field is downloaded. Let's review one example:

1. You open your NoteSelf wiki. All the skinny tiddlers are loaded from the local DB. Those tiddlers are not completed, they miss the text field.
2. You add some tiddler with some text. That tiddler is saved to the local database and to the wiki store (currently loaded wiki)
3. You save the wiki. What you will get is a TW with your tiddlers for this editing sessions and maybe those that you have opened for reading.

That was news to me. Thanks for explaining it so clearly.

 

To be honest, I can easily fix this situation by returning the FAT(full) tiddlers when skinny ones are requested, but the convention is to not do that. Also it could important impact on memory usage and performance because all the tiddlers would be loaded into memory. This may not be very important on desktops, but it is on mobiles and slow computers where TW behaves very slowly. 

I agree always loading the full tiddlers would not be the best default. 

Maybe I can include it as a configurable option and let every user chose what they want.

Or make the functionality for filling all the tiddlers available on demand. That way I can create my own save button which fills all the skinny tiddlers and then calls the normal save functionality. Then if I wanted to return to mostly skinny tiddlers again, just refresh the page.

Brian 

[Brian Theado]

Danielo,

On Tue, Aug 23, 2016 at 7:52 PM, Brian Theado <brian....@gmail.com> wrote:

[...] 

Only the data that is currently loaded in memory is included. What does this mean? By default, when tiddlywiki uses a sync-adaptor it first requests to it all skinny tiddlers. Those are tiddlers without the text field. When that tiddler is opened then the text field is downloaded. Let's review one example:

1. You open your NoteSelf wiki. All the skinny tiddlers are loaded from the local DB. Those tiddlers are not completed, they miss the text field.
2. You add some tiddler with some text. That tiddler is saved to the local database and to the wiki store (currently loaded wiki)
3. You save the wiki. What you will get is a TW with your tiddlers for this editing sessions and maybe those that you have opened for reading.

That was news to me. Thanks for explaining it so clearly.

I now have some evidence there is more to the story than what you explained. I was stepping through the code in the debugger to help me learn about the syncadapter functionality in general and the pouchdb adapter in particular. I had a breakpoint set in PouchAdaptor.prototype.loadTiddler and when I hit the save button, I saw the load tiddler was triggered for every tiddler I had stored in pouchdb. Even the ones which were not open and I hadn't opened. But before the asynchronous loading could take place, I saw the html file appear in the downloads status bar at the bottom of my chrome window. I guessed that the tiddlers got loaded, but only after the download file was generated. And I figured if they all loaded, then if I hit the save button a second time, then the download would this time contain the bodies of the tiddlers. I did a diff of the first download against the second and found that I was right!

If I'm not missing anything, then a full download can be had by clicking save, then waiting long enough for the tiddler bodies to be asynchronously loaded, then clicking save again. Not the most user friendly, but it seems to work.

Here's how you can duplicate my test:

1. open a new noteself database instance
2. create a few tiddlers with some body text
3. close all tiddlers
4. refresh the page (no open tiddlers = no tiddler body loading)
5. click the save button
6. look in the console for the "PouchAdaptor: Retrieving tiddler" message
7. click the save button a 2nd time
8. compare the two html files, noticing that the 2nd one has the tiddler bodies and the first does not.

Brian

[Danielo Rodríguez]

Hello Mark S.

El miércoles, 24 de agosto de 2016, 1:29:05 (UTC+2), Mark S. escribió:

Images can fill up TW pretty fast, slowing it down to a crawl.

That's because all images bodies are included as part of the TW. My sync adaptor only load in memory the tiddlers that are requested/opened by TW, so that overload does not happens.

 

If you want to include higher-resolution images (think maps) then you really need to park them outside TW.

Have you thought about using SVG for that? They are just plain text and has infinite resolution (literally). What do you mean with think maps?

 

My external image collection is just about 50megs on my main TW.

I didn't tried myself, but there are tests around there of PouchDB storing data up to 1GB without problems.

 

How do attachments work? I'm guessing they use binary as storage rather than the puffy base64.

I'm not sure because I haven't investigated it deeply yet, but I think it is like you said.

 

That's an improvement, but there's nothing that will make 1meg jpeg take up less than 1mb without loss of quality or resolution.

 Indeed. But, as I have asked before, what kind of images are you talking about?

[Mark S.]

Hello Danielo,

On Wednesday, August 24, 2016 at 4:28:09 AM UTC-7, Danielo Rodríguez wrote:

 

> If you want to include higher-resolution images (think maps) then you really need to park them outside TW.

>> Have you thought about using SVG for that? They are just plain text and has infinite resolution (literally). What do you mean with think maps?

 

I meant that maps are a good example of where high-resolution images are required. If there is a source for maps in SVG that would be great. I only know about screen grabs. However, recently I needed high-resolution photographs of an area that had trails, but not roads.

If your system with "lazy loading" actually works, then that would be sensational. I say "if" because lazy loading on my system with node.js doesn't work (the tiddler never populates).

Does the skinny-tiddler work even when the data is local?

I'm just trying it (loading NoteSelf) now with locally added images of about 2megs each. It's working nicely, but there is one little bug. The images don't have a creation date field, and so don't show up in the "recent" tab.

 
I'll have to play with this some more. Have you tried it with setting up your own couchdb server? I ask, because one thing that seems to be lacking is the ability to synch without going through a private service like Cloudant. If I'm too paranoid to use Evernote, I might also be too paranoid to use Cloudant. Yes, I know I can export and re-import, but if I have a 50 Meg TW file packed with images, the transfer process will be too long to be practicable.

I just tried dragging/dropping a PDF. It imported it as an octet stream that doesn't render. Is there a way to bring in PDFs ?

If I discontinue using NoteSelf on a particular machine or browser, how do I delete the associated data?

It does look promising!

Thank you!
Mark

[Danielo Rodríguez]

Hello Mark S.

 

 

I meant that maps are a good example of where high-resolution images are required. If there is a source for maps in SVG that would be great.

I'm not sure of what kind of maps you need (I mean, detail) but there are several open source maps sources. OpenStreetMaps is one of them and they allow you to export the map as SVG. Free SVG maps is another option. As I said, there are plenty of them

 

If your system with "lazy loading" actually works, then that would be sensational. I say "if" because lazy loading on my system with node.js doesn't work (the tiddler never populates).

Well, as far as I have tested it is working. Since when it's your node.js failing? I have reported a bug on 5.1.13 which could be the root cause: 

https://github.com/Jermolene/TiddlyWiki5/issues/2514

It has been patched and should be working now. NoteSelf uses a version patched by me until the 5.1.14 released, so yes, it is working.

 

Does the skinny-tiddler work even when the data is local?

Yes. The architecture is server-client, even tough the database is local TW sees it as a remote DB and acts consequently.

I'm just trying it (loading NoteSelf) now with locally added images of about 2megs each. It's working nicely, but there is one little bug. The images don't have a creation date field, and so don't show up in the "recent" tab.

That's a tiddlywiki bug, so it should happen on every tiddlywiki you use. Take a look at https://github.com/Jermolene/TiddlyWiki5/issues/2530

 

 
I'll have to play with this some more. Have you tried it with setting up your own couchdb server? I ask, because one thing that seems to be lacking is the ability to synch without going through a private service like Cloudant. If I'm too paranoid to use Evernote, I might also be too paranoid to use Cloudant.

I didn't tried to setup a couchdb server myself, but there are some users here that have tried and they succeed. NoteSelf should be compatible with any couchdb server, that includes cloudant but it is not limited to it.

 

I just tried dragging/dropping a PDF. It imported it as an octet stream that doesn't render. Is there a way to bring in PDFs ?

I didn't tried. I have to investigate it, thanks for reporting.

 

If I discontinue using NoteSelf on a particular machine or browser, how do I delete the associated data?

There is a tab under control panel labeled TiddlyPouch. Inside that tab there is another tab labeled database. There you can destroy the local database.

 

It does look promising!

Thank you!

Thank you Mark! Please note that this project it's still in beta, so please don't put any critical information on it. But, if you can try it's performance for your use case, which includes big amounts of data, it would be very useful to the project.

[Danielo Rodríguez]

Hello Brian,

El miércoles, 24 de agosto de 2016, 4:31:51 (UTC+2), Brian Theado escribió:

That was news to me. Thanks for explaining it so clearly.

You're welcome!

 

I now have some evidence there is more to the story than what you explained. I was stepping through the code in the debugger to help me learn about the syncadapter functionality in general and the pouchdb adapter in particular. I had a breakpoint set in PouchAdaptor.prototype.loadTiddler and when I hit the save button, I saw the load tiddler was triggered for every tiddler I had stored in pouchdb. Even the ones which were not open and I hadn't opened. But before the asynchronous loading could take place, I saw the html file appear in the downloads status bar at the bottom of my chrome window. I guessed that the tiddlers got loaded, but only after the download file was generated. And I figured if they all loaded, then if I hit the save button a second time, then the download would this time contain the bodies of the tiddlers. I did a diff of the first download against the second and found that I was right!

Very good catch! You have spotted a very interesting behavior. That happens because the lazy-loading mechanism is asynchronous, and the save mechanism is synchronous. When you click save the saver selects a template ($:/core/saveAll in this case) and tries to render it. Rendering the template triggers the lazy-loading mechanism for all the tiddlers, but because the synchronous nature of the rendering mechanism it does not waits the lazy-loading mechanism to complete and triggers the download with the available data at that moment. The lazy-loading mechanism continues working in the background and eventually completes, that's why when you click the saving button a second time you get a complete wiki with all the tiddlers.

If I'm not missing anything, then a full download can be had by clicking save, then waiting long enough for the tiddler bodies to be asynchronously loaded, then clicking save again. Not the most user friendly, but it seems to work.

 

Yes, that is definitely a workaround, but there is no way (currently) to know how long you have to wait. It can vary from the order of milliseconds to a couple of seconds, depending on the database size. To be honest, probably waiting a second will be enough for 99% of the times.

Here's how you can duplicate my test:

1. open a new noteself database instance
2. create a few tiddlers with some body text
3. close all tiddlers
4. refresh the page (no open tiddlers = no tiddler body loading)
5. click the save button
6. look in the console for the "PouchAdaptor: Retrieving tiddler" message
7. click the save button a 2nd time
8. compare the two html files, noticing that the 2nd one has the tiddler bodies and the first does not.

Brian

Thank you very much for your report Brian. I will think in ways around this. I have opened a bug on Tiddlywiki repository:

https://github.com/Jermolene/TiddlyWiki5/issues/2552

 

[Mark S.]

I made a tiddler earlier in the day. Just now I opened it up, changed the title. Now I'm getting messages like:

syncer-browser - 21:15:07 24 8 2016

Sync error while processing 'Quick Note': {"status":409,"name":"conflict","message":"Document update conflict"}

when I open it up to edit it I get:

syncer-browser - 21:17:27 24 8 2016

Sync error while processing 'Draft of 'Quick Note: {"status":409,"name":"conflict","message":"Document update conflict"}

After reloading the TW, the note is gone.

HTH

Mark

[Mark S.]

Although I like the skinny-tiddlers idea, I notice that if I search for something in text that is not in the current story it won't be found.

I suppose a long term solution would be a revised search engine that searches the text field back in the database.

The work-around for the present would be to make sure that any important key words are in the title.

Thanks!
Mark

[Danielo Rodríguez]

El jueves, 25 de agosto de 2016, 6:19:40 (UTC+2), Mark S. escribió:

I made a tiddler earlier in the day. Just now I opened it up, changed the title. Now I'm getting messages like:

syncer-browser - 21:15:07 24 8 2016

Sync error while processing 'Quick Note': {"status":409,"name":"conflict","message":"Document update conflict"}

when I open it up to edit it I get:

syncer-browser - 21:17:27 24 8 2016

Sync error while processing 'Draft of 'Quick Note: {"status":409,"name":"conflict","message":"Document update conflict"}

After reloading the TW, the note is gone.

Hello Marks S.

Thank you very much for the report, I was aware of the bug but not for tiddler renaming. 

This is, in my opinion, a bug caused partially by TW and partially by Pouch, see:

https://github.com/Jermolene/TiddlyWiki5/issues/2549

and

https://github.com/pouchdb/pouchdb/issues/5600

I'm not sure if I should patch it or wait until those are fixed. Probably I would patch it.

This could happening in the following situations:

• Tiddler rename
• Tiddler clone
• Tiddler created from template

Just to be curious, in case of rename what do you expect: to start from scratch without revision history or to keep it ? Thanks.

[Danielo Rodríguez]

El jueves, 25 de agosto de 2016, 17:22:29 (UTC+2), Mark S. escribió:

Although I like the skinny-tiddlers idea, I notice that if I search for something in text that is not in the current story it won't be found.

Yes I know. I have suffered it too.

 

I suppose a long term solution would be a revised search engine that searches the text field back in the database.

That is my plan. I want to intercept searches and perform then on the database. Maybe creating a different mechanism is required because searching the database on every keystroke sounds as too much overload.

 

The work-around for the present would be to make sure that any important key words are in the title.

In the title, the tags or any field except the text field. The last option (any other field) requires an special search or a filter search but can be very useful. Someone has created a "compress" algorithm that puts on a field all the unique words that are longer than 3 characters. That could be a good hack to think about.

 

Thanks!
Mark

[Mark S.]

Wait -- do tiddlers have a revision history in your edition, and, if so, how do we access prior versions?

Thanks!
Mark

[Danielo Rodríguez]

El jueves, 25 de agosto de 2016, 22:57:50 (UTC+2), Mark S. escribió:

Wait -- do tiddlers have a revision history in your edition, and, if so, how do we access prior versions?

Hello Mark. There is no UI for fetching revisions at the moment, but is one of the key features that I want to include.

To get the revisions of a tiddler you can open the console and paste the following command:

$tw.TiddlyPouch.database.get("$:/StoryList",{open_revs:"all"}).then(console.log.bind(console))

 You will see an array of objects on the console. You can navigate through them. Check the fields property to see previous versions of the tiddler.

Regards

[Danielo Rodríguez]

Dear Mark S.

The previous command will only give you leaf revisions (revisions in conflict). That's why I used story list. To get revisions of regular tiddlers a more complex command would be required. Let me know if you are interested anyway.

[Danielo Rodríguez]

El jueves, 25 de agosto de 2016, 6:19:40 (UTC+2), Mark S. escribió:

I made a tiddler earlier in the day. Just now I opened it up, changed the title. Now I'm getting messages like:

syncer-browser - 21:15:07 24 8 2016

Sync error while processing 'Quick Note': {"status":409,"name":"conflict","message":"Document update conflict"}

when I open it up to edit it I get:

syncer-browser - 21:17:27 24 8 2016

Sync error while processing 'Draft of 'Quick Note: {"status":409,"name":"conflict","message":"Document update conflict"}

After reloading the TW, the note is gone.

Dear Mark S.

I just released a patch that fixes this problem. Could you please take a look and let me know if it works for you too? Make sure the plugin version is 0.11.4

Regards.

[Mark S.]

First, let me explain that I updated my TW by dragging/dropping your new plugin into NoteSelf. Then I created a new tiddler called "Change my name". Then (after the 'save' check had flashed to gray) I reloaded. I verified that the plugin was still 11.4. Then I edited "Change my name", and before I had even done anything to the title, I got this error:

Sync error while processing 'Draft of 'Change my name:

{"status":409,"name":"conflict","message":"Document update conflict"}

I dismissed the error message and changed the title and saved, getting two more error messages. Then I reloaded ... and the retitled tiddler was gone.

I just realized that my testing might be unfair, since I'm using Pale Moon. Pale Moon is so much like FF, I forget that it's not 100% FF. I'll have to see if I can export my work over to FF.

Mark

[Mark S.]

Downloading a new copy from your site, and importing the data from my other NS, it might be allowing me to rename OK.

But there's another problem. I exported using the [!is[system]sort[title]] filter. It made a JSON file 10 megs or so. When I re-imported them into the new NS, I found that *all* of the tiddlers I had made in the last few days (only 3 actually) with text were empty, one of the images I had brought in was empty, and of the historic tiddlers there was some hit-and-miss. Perhaps I need a routine to force open all the tiddlers before export? In some cases this will be too much, I'm sure. If you had a Gig of info in there, you couldn't have them all open in your story without crashing.

Thanks!
Mark

[Danielo Rodríguez]

Dear Mark S.

Thank you for trying it out.
I'm not very confident that TW's plugin update mechanism work reliably with my plugin.
Would you mind to try it out with the online version of Noteself?
Noteself.github.io/online

Regards

[Mark S.]

Testing your online version for Tiddler Renaming (Using pale moon at the moment). Tiddler renaming seems to work.

The "delete" function may have a problem. I loaded the database with JSON data and then decided to delete it. I brought up the dialog and typed in "delete". Then I had a second thought and cancelled. But the data deleted anyways.

I'm still having trouble exporting data. I wrote a list loop that opened and closed all the tiddlers. Then I tried to export using the filter operator. It was hit-or-miss. Some items exported completely, others were empty (or skinny). For testing purposes, being able to export the current large (12mb) dataset accurately would be useful.

Thanks!
Mark

[Danielo Rodríguez]

Hello Mark S.

I'm sorry about the bug on the error mechanism. I have to review how the prompt works, it is a core feature and maybe I'm using it wrong. Until I fix it deleting the word "delete" will cancel the deletion.

Could you please explain how does your loop work? Having the tiddlers loaded on the wiki should allow you to reliably export them. This has nothing to do with the sync adaptor, if TW has the entire tiddler it will not request it to the sync adaptor.

I just released a new version. It includes a very basic management of revisions. It would be ideal if your test it and report what you miss on the revisions management.

Regards

[Danielo Rodríguez]

I forgot to mention that the revision management is inside the information area of each tiddler. Let me know if Yo have any problem finding it

[Mark S.]

This was the code I used, hoping to pull in the tiddlers. I could have left the tiddlers open, but wanted a more realistic situation. Opening all the tiddlers in a large TW would probably bring the whole system to its knees.

Mark

<$button>Blink em
<$list filter="[!is[system]sort[title]] -[[Open and close]]">
<$action-navigate $to=<<currentTiddler>> />
<$action-sendmessage $message="tm-close-tiddler" $param=<<currentTiddler>>/>
</$list>
</$button>

[Danielo Rodríguez]

Dear Mark S.

The problem with the 'delete' prompt was a tiddlywiki bug. I have reported it, and it is now fixed

https://github.com/Jermolene/TiddlyWiki5/issues/2561

This means that in the next TW version this will be fixed, and therefore in the next release of NoteSelf.

[Tobias Beer]

Hi Danielo,

Thanks for your work on this, looks highly interesting and promising.

I have a few simple to ask but possibly hard to answer questions
as to what NoteSelf can, cannot or may eventually do:

• Do I (always) need to login or can the connection be (safely/publicly) pre-configured (hard-wired) in a wiki?
  
  
• Asked differently, can I connect to a pouch-db in a kind of read-only "sync"-mode (w/o login)?
  
  
• Can I connect a single wiki to two different pouch-dbs?
  
  
• Can two different wikis connect to a single pouch-db?
  
  
• If yes, can I connect one wiki read-only (no login) and the other read-/write (login)?

The eventual goal of the above exercise would be
to allow one wiki to be used and exposed as a website and styled as such,

loading content read-only while displaying it in a more minimal, read-only manner,

and another wiki being the backend for editing (r/w)

with loads more plugins and showing the whole editing experience.

The "backend" could simply be a slightly different wiki, template-wise,
but still connect to the same pouch-db, but w/ login to get full r/w access
and then all the editing capabilities around the content.

Alternatively, there could be a way to use the backend for editing and
then some import mechanism (in another wiki) to pull the content-contents
and then publish an update as a publicly exposed website.

If you tell me there's a way for NoteSelf to achieve the above,
you will definitely have me hooked.

Best wishes,

Tobias.

[Danielo Rodríguez]

El martes, 30 de agosto de 2016, 7:32:45 (UTC+2), Tobias Beer escribió:

Hi Danielo,

Hello Tobias,

 

Thanks for your work on this, looks highly interesting and promising.

Thank you for your interest

 

I have a few simple to ask but possibly hard to answer questions

Let's see if I can get you in...

 

• Do I (always) need to login or can the connection be (safely/publicly) pre-configured (hard-wired) in a wiki?
  

That depends on the server configuration (see below). NoteSelf is not opinionated about the server configuration, it just tries to adapt to it.

 

• Asked differently, can I connect to a pouch-db in a kind of read-only "sync"-mode (w/o login)?
  

Again, this depends on the server. Please take a look below for some receipts

 

• Can I connect a single wiki to two different pouch-dbs?
  

You can have as many local databases as you want and you can connect to as many remote servers as you want. Normally each local database has a server attached to it, but you can change it at any time. So you can have both, several local databases connected to several remotes, or one single database and change to where it connects to.

 

• Can two different wikis connect to a single pouch-db?
  

I will take for granted that with "pouch-db" you refer to a remote couchdb server. The answer is YES, that's te point of the whole thing.

 

• If yes, can I connect one wiki read-only (no login) and the other read-/write (login)?

Yes you can, as long as the server allows this configuration. 

Here are some authentication recipes for couchdb. They include the use-case for each one, so it's pretty easy to see if it could fit your needs or not.

https://github.com/nolanlawson/pouchdb-authentication#couchdb-authentication-recipes

 

The eventual goal of the above exercise would be
to allow one wiki to be used and exposed as a website and styled as such,

loading content read-only while displaying it in a more minimal, read-only manner,

and another wiki being the backend for editing (r/w)

with loads more plugins and showing the whole editing experience.

That is a very tempting configuration. Probably it would be better to use Noteself in the "master" wiki and use a more light-weight plugin on the client one.

 

If you tell me there's a way for NoteSelf to achieve the above,
you will definitely have me hooked.

NoteSelf is intended to be a whole TW edition, focused on be some kind of Evernote replacement. Currently it's composed of a single plugin: TiddlyPouch, that brings the required sync capabilities. I have plans to create other NoteSelf specific plugins that will be part of NoteSelf edition. If One of the plugins fulfills your requirments you are not forced to take the full NoteSelf edition.

Regards.

[Danielo Rodríguez]

I just released a new update, version 0.13.2

This version includes a better UI for revision management:

• Revisions are displayed as tabs inside the same tiddler
• There is a button to open revisions tab of the info tiddler
• That button loads the revisions before opening the revisions tab.

Some fine-tuning is needed, but I want to have some user feedback before I continue developing the UI.

As usual, check at noteself.github.io

Regards

[Josiah]

Ciao Danielo

FYI, there is an exchange I just had with @Jermolone (Jeremy Ruston) on Twitter about SelfNote ...

Starts here: https://twitter.com/TiddlyTweeter/status/771634295353987073

Best wishes
Josiah

[Danielo Rodríguez]

El viernes, 2 de septiembre de 2016, 12:18:25 (UTC+2), Josiah escribió:

Ciao Danielo

FYI, there is an exchange I just had with @Jermolone (Jeremy Ruston) on Twitter about SelfNote ...

Starts here: https://twitter.com/TiddlyTweeter/status/771634295353987073

Hello Josiah,

That's a very daunting conversation. Seems to leave apart all NoteSelf features and focuses only on the worst part.

First of all, it is not saved to local storage, it is persisted on IndexedDB 

localStorage, or more accurately DOM Storage, was designed for smaller amounts of data. It's essentially a strings only key - value storage, with a simplistic synchronous API. 

indexedDB, on the other hand, was designed to work with significantly larger amounts of data. First, in theory, it provides both a synchronous and an asynchronous API. indexedDB's additional features could help you store all the data you need for your application to work offline.

Regarding "leaving information behind"

Serious privacy issues with leaving user data locked up the browser without their knowledge 

NoteSelf it's very clear about where information is stored, and provides SEVERAL EASY ways of deleting the information you store on it. Not to mention that leaving information on the browser is as insecure as leaving them on the filesystem.  

It's a bit sad that you leave apart all this features:

• Synchronize between multiple devices: Mobile, tablet, computer...
• Revision handling: store several versions of the same note and navigate through them.
• Two way synchronization with any remote CouchDB server
• Work offline, sync when you get back online
• Work totally offline if you want
• Works out of the box on your browser, does not require any plugin
  
• Plug and Play, No installation or configuration required.

and focus on a technology that is not even being used.

[Jeremy Ruston]

Hi Danielo

That's a very daunting conversation. Seems to leave apart all NoteSelf features and focuses only on the worst part.

Not at all! In one of the messages I said that the CouchDB support was terrific but that I wasn’t a fan of local storage.

First of all, it is not saved to local storage, it is persisted on IndexedDB 

I was using “local storage” in lower case to cover all kinds of storage in the browser. There’s little practical difference for an end user between IndexedDB and localStorage.

Regarding "leaving information behind"

Serious privacy issues with leaving user data locked up the browser without their knowledge 

NoteSelf it's very clear about where information is stored, and provides SEVERAL EASY ways of deleting the information you store on it. Not to mention that leaving information on the browser is as insecure as leaving them on the filesystem.  

That comment was with respect to a different comment of mine, to the effect that I could see a role for local storage as a temporary backup in case saving fails.

It's a bit sad that you leave apart all this features:

But literally all of those features are part of CouchDB, and hence covered by my approval in the original tweet.

and focus on a technology that is not even being used.

Part of the issue is that part of the reaction to NoteSelf has actually been reaction to seeing browser local storage used for the first time. To me, that’s really an irrelevant distraction from what’s cool about NoteSelf.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/74489054-6fe4-41db-9f21-2a038e4c39de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Josiah]

Ciao Jeremy & Danielo

> Part of the issue is that part of the reaction to NoteSelf has actually been reaction to seeing browser local storage used for the first time. To me, that’s really an irrelevant distraction from what’s cool about NoteSelf.

I don't think its entirely irrelevant. One of the things I like about it is that it saves locally in a uniform way--in the browsers I have tested so far. As I commented previously having ONE method, not browser plugin-in dependent, for local save, is, I think, very useful to see working.

I do think there are issues with local storage. Which can likely be overcome. I'm less concerned about privacy than portability.

Best wishes
Josiah

[Jeremy Ruston]

Hi Josiah

On 2 Sep 2016, at 16:13, Josiah <prog...@assays.tv> wrote:

Ciao Jeremy & Danielo

> Part of the issue is that part of the reaction to NoteSelf has actually been reaction to seeing browser local storage used for the first time. To me, that’s really an irrelevant distraction from what’s cool about NoteSelf.

I don't think its entirely irrelevant.

Perhaps we can agree that it is at least orthogonal. If all that one wanted was local storage of tiddlers then that could be added much more simply than what you see in NoteSelf. The point of NoteSelf is to bring up the CouchDB interface; it would still be a desirable thing to do, even without the local storage features.

One of the things I like about it is that it saves locally in a uniform way--in the browsers I have tested so far. As I commented previously having ONE method, not browser plugin-in dependent, for local save, is, I think, very useful to see working.

I do think there are issues with local storage. Which can likely be overcome.

Really? The issues with browser local storage seem to me to be pretty fundamental. As I said, it’s brittle and opaque. The user doesn’t have control over the lifetime of the data, they can’t inspect it, and they can’t export it. That lack of control is by design because the mechanism is designed for a different usecase: as a browser-side cache for server applications that will ultimately persist their data at the server. The browser manufacturers have zero interest in using local storage as a persistent, reliable storage locals.

Perhaps it’s just me: local storage has been around for perhaps 10 years, so I’ve had plenty of time to absorb the capabilities and limitations.

I'm less concerned about privacy than portability.

As I said to Danielo, it’s only when considering using local storage as a default cache that the privacy issues arise. It is indeed all about control.

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ef38dd46-cadd-4f53-914f-0f46fb4fa7e0%40googlegroups.com.

[Josiah]

Ciao Jeremy

I think this is worth hitting on a bit more.

As you likely know I'm more of a standard user than a person with insight into the history of tech.

I'm making, perhaps, a very naive point  ... but I like the fact that NoteSelf can save locally in different browsers without the user having to install anything additional.

Best wishes
Josiah

[Jeremy Ruston]

Hi Josiah

I'm making, perhaps, a very naive point  ... but I like the fact that NoteSelf can save locally in different browsers without the user having to install anything additional.

I get that, and I’m acknowledging that local saving is a neat trick, but pointing out that the flaws, in my view, make it unsuitable for serious work. I understand that you’d like to see a way to save changes without installing anything, but I don’t think local storage is the answer you’re looking for.

Anyhow, of course, the default fallback saver doesn’t require a plugin, and works in almost all browsers, and is part of the standard HTML5 specification…

The trouble with trade-offs is that you need to trade them off :)

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0a695b88-b7eb-40ca-b0d2-26acf8a920a9%40googlegroups.com.

[Danielo Rodríguez]

I'll leave apart the discussion about LocalStorage, I want to get back to it tomorrow when I have more energy.

Today I want to notify another update to NoteSelf. Here is the changelog of most noticeable features since the latest version I have announced:

0.14.2

• DefaultTiddlers is loaded at startup along with StoryList
  
• Dowload saver that downloads all the tiddlers contained on the current db as JSON
  

0.14.1

• Revisions are validated before saving

0.14

• First version of module loader

0.13

• Better UI for revision handling
  • See revisions as tabs of the current tiddler
  • Open several previous revisions to check them all
  • Revisions names are shorted for readability

0.12.1

• Set the database name as subtitle
  • Click on it to open the control panel on the database selection tab

0.12

• Basic revision handling
• Get all revisions of a tiddler 
• Get revision of a tiddler

Some of the new features that I want to highlight are:

Dowload saver that downloads all the tiddlers contained on the current db as JSON

This solves one the main user's concerns. Previously the results from clicking the download button were unpredictable. The new saver module packs all the tiddlers into a JSON file that can be later imported into ANY wiki.

Revisions are validated before saving

This will allow you to safely import tiddlers that have been lying around in other sync mechanism (node js edition, for example).

Better UI for revision handling

This is probably one of my favourites. I have been working hard to provide a good-looking and easy to use user interface for revision checking. There are more things that I want to implement, but for the moment I'm pretty happy with it.

As usual, you can check it out at https://noteself.github.io and https://noteself.github.io/online/

Feedback is welcome!

Regards

[Brian Theado]

Danielo,

On Sat, Sep 3, 2016 at 7:41 PM, Danielo Rodríguez <rdan...@gmail.com> wrote:

Some of the new features that I want to highlight are:

Dowload saver that downloads all the tiddlers contained on the current db as JSON

This solves one the main user's concerns. Previously the results from clicking the download button were unpredictable. The new saver module packs all the tiddlers into a JSON file that can be later imported into ANY wiki.

I tested this a little...enough to see that the skinny tiddlers did indeed fully download when I clicked save and that I could import that .json file into a new tiddlywiki.

 

Better UI for revision handling

This is probably one of my favourites. I have been working hard to provide a good-looking and easy to use user interface for revision checking. There are more things that I want to implement, but for the moment I'm pretty happy with it.

This looked good. I could open two revisions and click back and forth between the tabs and visually see the change between the two tiddlers. As I played with it, I wondered what the edit button would do when I have the tab of another version open. It looks like it edits the current version. That makes sense.

I happen to try to open the online noteself from my phone on android chrome and android firefox. In both cases the page loaded to a blank browser screen. Probably it didn't load due to some error?

Brian

 

[Danielo Rodríguez]

Hello Brian

> I happen to try to open the online noteself from my phone on android chrome and android firefox. In both cases the page loaded to a blank browser screen. Probably it didn't load due to some error?

I tested it both on my phone (android 6) and my tablet (android 4.4) and works fine. Which android version are you using? Which chrome version? Regards

[Omar Gonsenheim]

Hola Daniello, 

I downloaded the empty.html file to my PC but my browser is not displaying anything, I tried with Chrome and Firefox without luck.

I checked the file with Notepad++ and the text is there. Any suggestions ?

I apologize in advance if this is a totally newbie question which I am ;-)

[Brian Theado]

Danielo,

Android version is 6.0.1 and chrome is 52.0.2743.98.

I just hooked up chrome remote debugger and it paused on this exception:

"TypeError: $tw.wiki.getTiddlerData is not a function

at _readConfigTiddler ($:/plugins/danielo515/tiddlypouch/startup/config.js:55:25) 

at eval ($:/plugins/danielo515/tiddlypouch/startup/config.js:213:27)" 

Console contains this:

TiddlyPouch:config: Initializing config module

TiddlyPouch:config: Config read from DB - ERROR 

{status: 404, name: "not_found", message: "missing", error: true, reason: "missing"}

Brian

[Brian Theado]

Looks like chrome remote debugger was overkill as it doesn't seem to be android specific. I just duplicated the issue on my desktop by opening https://noteself.github.io/online/ in an incognito window.

[Rizwan Ishak]

Reporting the same issue
Browsers: Firefox 48, Chrome v53.
OS: Linux Mint.

[Jason Maskell]

I'm having the same issue to. From the console:

TiddlyPouch:config: Initializing config module

TiddlyPouch:config: Config read from DB - ERROR n
Uncaught (in promise) TypeError: $tw.wiki.getTiddlerData is not a function(…)
Error in event handler for (unknown): TypeError: Cannot read property 'shortcutLock' of undefined
    at chrome-extension://oknpjjbmpnndlpmnhmekjpocelpnlfdi/js/content.js:32:32

I disabled Readability extension - now the error is:

TiddlyPouch:config: Initializing config module

TiddlyPouch:config: Config read from DB - ERROR n {status: 404, name: "not_found", message: "missing", error: true, reason: "missing"}
Uncaught (in promise) TypeError: $tw.wiki.getTiddlerData is not a function(…)

Firefox with no extension installed (fresh install):

TiddlyPouch:config: Initializing config module $:/core/modules/utils/logger.js:30:10
TiddlyPouch:config: Config read from DB - ERROR Object { status: 404, name: "not_found", message: "missing", error: true, reason: "missing", stack: "" } $:/core/modules/utils/logger.js:30:10
TypeError: $tw.wiki.getTiddlerData is not a function empty.html%20line%208794%20%3E%20eval:55:16

I'm running: 

Windows 7 

Google Chrome Version 51.0.2704.106 m (64-bit)

Firefox 47.0.1

[Danielo Rodríguez]

Hello everyone,

Thanks to all for your feedback. I really appreciate the extracted logs. Thank you for those that even took the time to connect the remote debugger.

I think this is an issue with newcomers. I mean, people that do not have an existing DB. In absence of database it reads the default configuration from a tiddler, and because recent changes this fails.

I'll fix it today and publish a release.

Thank you very much

[Danielo Rodríguez]

Hello everyone,

I just published a hotfix. It is already available at noteself.github.io

The version should be 0.14.3+1

If it fails, you can open the console and check the version with:

$tw.wiki.getTiddler("$:/plugins/danielo515/tiddlypouch").fields.version

if it is not 0.14.3+1 make sure to refresh the page or clear the cache.

I'm happy that this bug showed up several users. I didn't know that there were so many people trying it!

Thanks all!

Regards!!!

[Rizwan Ishak]

Reporting: Solved
Brwsers: Firefox 48, Chrome 53
OS: Linux Mint

[Danielo Rodríguez]

Thank you Rizwan Isaak for the report!

[Anton Aylward]

Oh, way cool!

I did a lot with the pre-5 TiddlyWiki in the way of custom templates, most specifically for writing a book.

Years ago there was the Jana Contact Manager, if anyone remembers that, which allowed, for example multiple addresses, history of workplaces for individuals, multiple phone associated with with individuals or addresses, and forward-backward likns to all this, for example who else worked at the company an individual worked at 5 years go.  Integrated dialer, fax, document management and call logging.

Massive overkill for most people, excessive for most salesmen, it was written originally for a political party for fundraising.
But it was a damn good contact manager, better than anything else I've come across on the PC or tablet's PlayStore.

The portability of NoteSelf makes me wonder about duplicating some of that functionality, thing like multiple addresses, other contacts, other phone numbers, something that doesn’t fit into a predetermined number of slots like on a ACT 5x8 card. 

Of course that needs the relational database :-)  Which is probably why I've had no success with doing with bare Tiddlers.
So this looks like it has potential.  Can I get closer to the database layer in my templates?

What does the group think?

[Tobias Beer]

Hi Anton,

Can I get closer to the database layer in my templates?

I think to model something more DB'ish in TiddlyWiki, what you want to do is create the smallest meaningful chunks of information possible with some well defined patterns like

• a tagging structure / tree
• I often have the following pattern:
• a tiddler called Contacts that all tiddlers have around the topic of "Contacts"
• a tiddler called Contact to that all tiddlers have that are Contacts
• that way I can separate records from any ideas around those records
  
• named fields with defined contents / content patterns
• tiddler naming conventions, e.g. certain prefixes, like #next, #urgent, #done, @call, @read, @buy, etc...
• similar to the $:/foo/bar/system structure
• tiddler templates
• or forms that allow you to create tiddlers of a given, well defined type

All of that, however, is plain TiddlyWiki, NoteSelf or not.

Best wishes,

Tobias.

[Carl Kibler]

I am thinking about a Cloudinary plugin which would grab inserted images (or other documents) and move them into my (free tier) personal Cloudinary account. Other services would work as well, just without the image-formatting tricks it provides. But they could be pushed to S3 and replaced with a link, for example. That would fit my particular use but of course break the one-file-for-all-data paradigm which many users do want. 

On Tuesday, August 23, 2016 at 5:29:05 PM UTC-6, Mark S. wrote:

Images can fill up TW pretty fast, slowing it down to a crawl. If you want to include higher-resolution images (think maps) then you really need to park them outside TW. My external image collection is just about 50megs on my main TW.

How do attachments work? I'm guessing they use binary as storage rather than the puffy base64. That's an improvement, but there's nothing that will make 1meg jpeg take up less than 1mb without loss of quality or resolution.

Thanks!
Mark

On Tuesday, August 23, 2016 at 4:11:34 PM UTC-7, Danielo Rodríguez wrote:

Hello Mark S.

Tiddlywiky, and by extension Noteself, supports images. I tried with some small images (50k) and I can embed them with no problems.

Worth mentioning that the database that I'm using supports attachments, which is a more efficient way of including binaries than using base64. It's not implemented yet but is on my list.

What problem do you see exactly?

[Lost Admin]

Dredging up the past here but I want to make sure I'm reading this correctly and that it is still true.

If I read the below correctly, Danielo is saying that to turn TiddlyWiki into NoteSelf, I can take the TiddlyPouch plugin out of NoteSelf and put it into an existing TiddlyWiki?

On Tuesday, August 30, 2016 at 5:22:21 PM UTC-4, Danielo Rodríguez wrote:

...

[Mark S.]

Is the configuration video supposed to be silent?

I checked on 2 browsers and there was no sound. I also ran a separate video to verify my drivers and sound were working.

Thanks,
Mark

[Danielo Rodríguez]

Hello Mark,
Yes it lacks sound. I want to add voice comments,but I'm still looking for a time slot.
Regards

[Danielo Rodríguez]

Hello Lost Admin,

That sentence is not 100% true at the moment.
NoteSelf has evolved and it now includes more plugins than just tiddlypouch. However, the minimum setup should be tiddlypouch + PouchDB (there is a plug-in created by me that includes it).
But probably the easier way is to download the empty edition of NoteSelf

[Mark S.]

Are there text instructions somewhere? I think text would be better than a soundless video. When you type in an address on screen, for instance, I have no idea if I'm literally supposed to type that address, or some other address of my own.

I seem to recall that Youtube has an editor that will let you overlay text on the screen. That might help to explain what you're doing.

Thanks,
Mark

[Lost Admin]

Where would I find that plug-in of yours?

I'm trying to understand how you did what you did, so it would be nice to see how it interacts with the vanilla tiddlywiki. I also want to integrate Ton's TopLeftToolbar so that the NoteSelf cloud/sync/login button appears on the top of the screen all the time but that's a different problem.

For some reason (I haven't figured out yet) if I copy the empty Noteself and run it from a local file, it never gets to the point where I can enter in the server/database/credentials.

Basically, I like what you are doing and I want to fiddle with it.

[Danielo Rodríguez]

El martes, 11 de julio de 2017, 19:50:03 (UTC+2), Lost Admin escribió:

Where would I find that plug-in of yours?

You can take a look at how NoteSelf is built. On the repository there is a jenkins file that downloads all the required plugins. In any case, you can find it on my personal github:  https://github.com/danielo515/tw5-pouchdb

 

I'm trying to understand how you did what you did, so it would be nice to see how it interacts with the vanilla tiddlywiki.

Hope you get fun with it.

 

I also want to integrate Ton's TopLeftToolbar so that the NoteSelf cloud/sync/login button appears on the top of the screen all the time but that's a different problem.

That is not a problem. You can add any plugin or customization to NoteSelf and it will work out of the box.

 

For some reason (I haven't figured out yet) if I copy the empty Noteself and run it from a local file, it never gets to the point where I can enter in the server/database/credentials.

How are you obtaining the empty copy of NoteSelf ? Have you configured the remote database ? Synchronization is not available until you configure it.

 

[TonyM]

I will second that,

NoteSelf seems promising but I would like just a little more text that describes its relationship to the Database, what it takes to host or create a database, I get the idea that the tiddlers will be stored in the database, thus accessed from alternate locations. does this mean the tiddlywiki file can be read only once connected to a database? For a mobile do we download the file and then connect it to the database? in which there is no local updates?

I am sure it is my own ignorance, but it feels the is much "written between the lines" in what information I can find.

Tony

[Lost Admin]

Danielo,

With regards to your last question:

Short-ish answer: http://thelostadmin.tiddlyspot.com/#6th%20July%202017

Longer-ish answer:

I've got a functioning CouchDB running in a VM on my laptop. I followed the CouchDB instructions for setting up a server. I also referenced your instructions video to check for any special settings that might be needed.

The VM is also running a web server (lighttpd). I copied your "online" Noteself to my web server and removed the google analytics plugin.

I had a bit of trouble sorting out CORS but after some reading and fiddling I got it to work.

I successfully created a test Noteself and confirmed:

 Login works, and is required

 My tiddlers are being stored in the couchdb (I manually looked into Couchdb to confirm)

 Updates to my tiddlers are being stored (I manually looked into Couchdb to confirm)

 If I wipe out browser data (so the local database is empty) I can restore from couchdb simply by putting the Noteself configuration settings back into my hosted Noteself. It reloads from the couchdb server.

Where I run into a problem:

 Adding TopLeftToolbar works but loads "funny".

Adding the $:/tags/TopLeftBar tag to $:/plugins/danielo515/tiddlypouch/ui/sync-flag doesn't stick when saving "the Noteself way". This is to let me put the sync flag on the top of the screen so I know if I'm logged-in and synced without having to open the side-bar. I haven't found a way to make it stick (yet).

Using the download link of Noteself to get the Noteself version of empty.html results scary errors. It flickers for about 30 seconds (not the normal nice scrolling "Loading" and then produces some error messages (Chrome):

Or RSOE (IE11)

[Danielo Rodríguez]

Hello Lost Admin

El miércoles, 12 de julio de 2017, 15:04:32 (UTC+2), Lost Admin escribió:

Danielo,

With regards to your last question:

Short-ish answer: http://thelostadmin.tiddlyspot.com/#6th%20July%202017

Seems that you have created a very complete and good manual on how to host your own CouchDB server and make NoteSelf work with it. I'll love to see such manual in a pull request to noteself.github.io repository, so other users can take advantage from it.

 

The VM is also running a web server (lighttpd). I copied your "online" Noteself to my web server and removed the google analytics plugin.

Is local storage important for you? I mean, using the browser as a temporary or intermediate memory. If not, you can save the lighttpd server using the other plugin I have mentioned before.

 

I successfully created a test Noteself and confirmed:

 Login works, and is required

 My tiddlers are being stored in the couchdb (I manually looked into Couchdb to confirm)

 Updates to my tiddlers are being stored (I manually looked into Couchdb to confirm)

 If I wipe out browser data (so the local database is empty) I can restore from couchdb simply by putting the Noteself configuration settings back into my hosted Noteself. It reloads from the couchdb server.

That's fantastic!

 

Where I run into a problem:

 Adding TopLeftToolbar works but loads "funny".

Adding the $:/tags/TopLeftBar tag to $:/plugins/danielo515/tiddlypouch/ui/sync-flag doesn't stick when saving "the Noteself way". This is to let me put the sync flag on the top of the screen so I know if I'm logged-in and synced without having to open the side-bar. I haven't found a way to make it stick (yet).

What do you mean by, "the Noteself way" ? If you mean letting it go to the pouchdb database and then load it from there, then it may not work. Only certain tiddlers are loaded before TW boots, and after TW boots it is responsibility of tiddlywiki to load the required tiddlers. This sometimes is noticeable by seeing how certain elements appear on the screen after some seconds. Is this happening to you or it is not loaded at all ?

In any case, if you want to "hardcode" your customizations into the html file (which seems convenient in your case) there is a button on the control panel for that matter. Please take a look at the following screenshot:

That button will download all the non user tiddlers into the html file that you can then host whenever you want. Give it a go and let me know how it goes.

 

Using the download link of Noteself to get the Noteself version of empty.html results scary errors. It flickers for about 30 seconds (not the normal nice scrolling "Loading" and then produces some error messages (Chrome):

Or RSOE (IE11)

Definitely I have to check it again, never happened to me before. 

[Danielo Rodríguez]

Hello TonyM

El miércoles, 12 de julio de 2017, 8:48:51 (UTC+2), TonyM escribió:

I will second that,

NoteSelf seems promising but I would like just a little more text that describes its relationship to the Database, what it takes to host or create a database, I get the idea that the tiddlers will be stored in the database, thus accessed from alternate locations. does this mean the tiddlywiki file can be read only once connected to a database? For a mobile do we download the file and then connect it to the database? in which there is no local updates?

I tried to be as clear as I could on the documentation. However, I have a big technical background and sometimes explaining things without technical details is hard to me, I'm open to any suggestion for improvement.

I think this is on the documentation but, let me try to explain it clearer: Accessing the online version will "install" NoteSelf in your browser. Tiddlers are saved on your browser's storage. If you don't have mobile connection, then no problem, there are stored on your local database and synced when you come back online. You don't have a remote database configured ? No problem, tiddlers are saved on your local database and when you add a remote one then they are synced to it.

NoteSelf is an offline first application, which means that it was built with offline functionality in mind, and it also works when it has connectivity. Hope this clarifies your doubts a bit more

[Lost Admin]

On Wednesday, July 12, 2017 at 10:28:58 AM UTC-4, Danielo Rodríguez wrote:

Hello Lost Admin

El miércoles, 12 de julio de 2017, 15:04:32 (UTC+2), Lost Admin escribió:

Danielo,

With regards to your last question:

Short-ish answer: http://thelostadmin.tiddlyspot.com/#6th%20July%202017

Seems that you have created a very complete and good manual on how to host your own CouchDB server and make NoteSelf work with it. I'll love to see such manual in a pull request to noteself.github.io repository, so other users can take advantage from it.

It isn't (yet) complete as far as covering what I want. I'll try to remember to let you know when it is so we can work to include it in the noteself github.

 

 

The VM is also running a web server (lighttpd). I copied your "online" Noteself to my web server and removed the google analytics plugin.

Is local storage important for you? I mean, using the browser as a temporary or intermediate memory. If not, you can save the lighttpd server using the other plugin I have mentioned before.

Local storage, as in the pouchdb that you have in Noteself, is important for what I'm trying to do. As is having a solid how-to make a complete solution. I'm working on taking your idea of Noteself being an alternative to evernote to the next step by documenting set-up of all of the needed infrastructure (web server couchdb, etc.). Okay, so my real goal is to make it easier to use tiddlywiki from my phone (tiddlyspot doesn't work offline) and sync with desktop (something screwy with tiddlywiki, tiddlyspot and browser cache).

I successfully created a test Noteself and confirmed:

 Login works, and is required

 My tiddlers are being stored in the couchdb (I manually looked into Couchdb to confirm)

 Updates to my tiddlers are being stored (I manually looked into Couchdb to confirm)

 If I wipe out browser data (so the local database is empty) I can restore from couchdb simply by putting the Noteself configuration settings back into my hosted Noteself. It reloads from the couchdb server.

That's fantastic!

 

Where I run into a problem:

 Adding TopLeftToolbar works but loads "funny".

Adding the $:/tags/TopLeftBar tag to $:/plugins/danielo515/tiddlypouch/ui/sync-flag doesn't stick when saving "the Noteself way". This is to let me put the sync flag on the top of the screen so I know if I'm logged-in and synced without having to open the side-bar. I haven't found a way to make it stick (yet).

What do you mean by, "the Noteself way" ? If you mean letting it go to the pouchdb database and then load it from there, then it may not work. Only certain tiddlers are loaded before TW boots, and after TW boots it is responsibility of tiddlywiki to load the required tiddlers. This sometimes is noticeable by seeing how certain elements appear on the screen after some seconds. Is this happening to you or it is not loaded at all ?

Yes, "the Noteself way" would be using pouchdb & couchdb.

 

In any case, if you want to "hardcode" your customizations into the html file (which seems convenient in your case) there is a button on the control panel for that matter. Please take a look at the following screenshot:

That button will download all the non user tiddlers into the html file that you can then host whenever you want. Give it a go and let me know how it goes.

I will give it a go. On a related note, do you think  you could add a "caption" and "description" field to the sync-flag tiddler so they show up properly in the settings?

[Danielo Rodríguez]

Hello Lost Admin

 

It isn't (yet) complete as far as covering what I want. I'll try to remember to let you know when it is so we can work to include it in the noteself github.

Sure, in any case please feel free to open an issue on github as a reminder, and maybe a PR a bit latter (when completed) 

 

Local storage, as in the pouchdb that you have in Noteself, is important for what I'm trying to do.

Then you made the right choice :)

 

I'm working on taking your idea of Noteself being an alternative to evernote to the next step by documenting set-up of all of the needed infrastructure (web server couchdb, etc.).

I don't see how that is taking the idea to the next step. Of course that is very needed information, and a very good addition, but the idea and it's objective stills the same.

 

Okay, so my real goal is to make it easier to use tiddlywiki from my phone (tiddlyspot doesn't work offline) and sync with desktop (something screwy with tiddlywiki, tiddlyspot and browser cache).

That was EXACTLY THE SAME goal that I was after when I started this project. At first I only wanted something that worked for me, that is how TiddlyPouch born, then I wanted to make it easier to use to a broader audience, and that is when NoteSelf born. As we are pursuing the same goals, I think it would be better if we can help each other, no matter if we do it on the same project or on different ones. The fact is the more people involved, the stronger the idea will become 

 

 On a related note, do you think  you could add a "caption" and "description" field to the sync-flag tiddler so they show up properly in the settings?

Sure, could you please open an issue on Github for it? 

[Lost Admin]

On Thursday, July 13, 2017 at 4:49:48 AM UTC-4, Danielo Rodríguez wrote:

Hello Lost Admin

... 

Okay, so my real goal is to make it easier to use tiddlywiki from my phone (tiddlyspot doesn't work offline) and sync with desktop (something screwy with tiddlywiki, tiddlyspot and browser cache).

That was EXACTLY THE SAME goal that I was after when I started this project. At first I only wanted something that worked for me, that is how TiddlyPouch born, then I wanted to make it easier to use to a broader audience, and that is when NoteSelf born. As we are pursuing the same goals, I think it would be better if we can help each other, no matter if we do it on the same project or on different ones. The fact is the more people involved, the stronger the idea will become 

 

 On a related note, do you think  you could add a "caption" and "description" field to the sync-flag tiddler so they show up properly in the settings?

Sure, could you please open an issue on Github for it? 

I'm not planning on spinning off a separate project so much as adding what I would like to have around NoteSelf. I expect working together would be the best way so long as what I do doesn't go against  your goals.

Also, I tend to be very slow at development as I have a busy life so I may only get a half day every couple of weeks to work on a project. I can read forums at work but to code requires free time. (and I'm not a very good coder). 

[TonyM]

Thanks Danielo,

I empathize with the difficulty of sharing technical knowledge when you are deeply involved with a technical solution.

What you have said helps. As I understand it then, The conceptual leap I needed to make here is that the data remains inside the browser,
but recorded on cache disk between sessions, across multiple sessions,
so to access your changes you need to use the same browser, and presumably the same profile or user in that browser.

However you can externalize your data by connecting to a Data base external to the browser, be that hosted locally or on the internet.
I will call this an "Established" instance of the NoteSelf TiddlyWiki.

Now, all that remains for me to wrap my head around is the online vs download method.

Presumably regardless of which you use, they both install "themselves" in the browser session where you open them,
This would then mean if you copied a local noteself file from one computer to another and thus/or open it in a different browser session, it would revert to a "fresh instance".

Presumably If I edit and save the local noteself file (with added content) using tiddlyfox, then move the file, it will start with the added content + a fresh instance of changes going forward.

In addition if connecting to an existing database the above "fresh instances" will in fact be "established instances"

Please tell me if I have got it right, and feel free to reuse my text to add to the documentation if required.

Regards
TonyM

[Danielo Rodríguez]

Hello TonyM

El viernes, 14 de julio de 2017, 6:15:26 (UTC+2), TonyM escribió:

Thanks Danielo,

I empathize with the difficulty of sharing technical knowledge when you are deeply involved with a technical solution.

Thank you :-)

 

What you have said helps. As I understand it then, The conceptual leap I needed to make here is that the data remains inside the browser,
but recorded on cache disk between sessions, across multiple sessions,
so to access your changes you need to use the same browser, and presumably the same profile or user in that browser.

That is an 100% accurate explanation

 

However you can externalize your data by connecting to a Data base external to the browser, be that hosted locally or on the internet.
I will call this an "Established" instance of the NoteSelf TiddlyWiki.

Cool name, and again, correct

 

Now, all that remains for me to wrap my head around is the online vs download method.

Presumably regardless of which you use, they both install "themselves" in the browser session where you open them,
This would then mean if you copied a local noteself file from one computer to another and thus/or open it in a different browser session, it would revert to a "fresh instance".

You got it!

 

Presumably If I edit and save the local noteself file (with added content) using tiddlyfox, then move the file, it will start with the added content + a fresh instance of changes going forward.

That is almost correct except for the tiddlyfox part. You should use the download saver. Well, if you are using tiddlyfox it should work too, but I never tried. Apart from that, the rest is absolutely correct.

 

In addition if connecting to an existing database the above "fresh instances" will in fact be "established instances"

Not sure to understand the difference between fresh and established instances, but all the rest you said was correct, so I will think this one too.

 

Please tell me if I have got it right, and feel free to reuse my text to add to the documentation if required.

As I said, you are right in the 99% for your assumptions. I'll love to find some time to integrate part of your explanation into the project. 

[Diego Mesa]

Hi Danielo,

Thanks for making and sharing this. I currently use a node installation of tw5 on my local machine, running out of a dropbox folder. I am most interested in adding the "revisions" feature to my tw5 installation. Do you have any information on that?

Thanks!

Diego

[Diego Mesa]

Can anyone point to a "revisions"-like plugin? Thanks!