[TW5] codemirror TW5 mode support?

[TheDiveO]

Is anyone thinking about adding a TW5 syntax highlighting mode to codemirror in a similar way codemirror supports a TWC mode?

Regards,
-- TheDiveO

[Ed Dixon]

Hi,

Yes, I do not know how CodeMirror was implemented in TWC (I will look) and I am new to TW5 in general, but I have this need as well for a project I am working on and would like to fully support all CM features (themes/ addons/ etc) either by extending the existing plugin, or creating a new one. I do not know if the current plugin was stripped of these features to reduce bloat for the majority of users who do not need them (which makes perfect sense) or if there are existing conflicts with these features and TW5 in general. I have not started building on this yet and this would be my first plugin / extension if successful. I do hope to add a configuration widget for the CM setup as well. I am looking forward to the video meeting with Mr. Ruston on Tuesday to gain more insight and get to know some of the team better here. I have watched all the previous video hangouts and reread the documentation several times and feel like I know many of you already! I hope you can make it.

Thanks,

[PMario]

Long post warning :)

Hi Ed,
I did codemirror for TWc.

Yes, I do not know how CodeMirror was implemented in TWC (I will look)

The TWc concept used to highlight the syntax in tiddler view mode, can't be used anymore with TW5.
CM for TWc implemented some parsing rules for eg: //{{{ javascript }}}//  or <!--{{{--> html <!--}}}-->... These wikitext rules are not there anymore in TW5

A TW5 tiddler has a type. eg: application/javascript which defines how the syntax should be highlighted. For javascript detection is easy. .. but eg:

CSS will be more complicated, because there are 2 type possibilities in TW:

a) type: text/css ... easy ... type can be used for highlighting
b) type: text/vnd.tiddlywiki  - tag: $:/tags/Stylesheet ... more complicated, since CSS code can include wikitext now. So what you need here is a mixed mode. Codemirror supports this, but it is more complicated, than "pure" modes. (There are at least 2 mechanisms in cm that could be used :)

There are several other edge cases eg: with svg. SVG has type: text/svg+xml   .. It can be viewed as an image (TW default). But if TW is used as a programming editor svg  also can be viewed as highlighted text but type has to be text/svg+xml. So a different "view type" has to be defined. ..

-------------

 I do not know if the current plugin was stripped of these features to reduce bloat for the majority of users who do not need them (which makes perfect sense)

The codemirror plugin was initially created by Jeremy, as a proof of concept. Then someone hardcoded several cm keyboard bindings (cm plugins) into the TW cm-plugin, to make it more useful: emacs, vim, sublime. The result is pure bloat. ... CM keyboard bindings are plugins for a reason. ... nobody needs them all.

There are also some addons: dialog, edit and search. ... imo they are ok, since the default browser search mechanism doesn't work with long code tiddlers.

So there are no conflicts. .. It's just size, that matters. ...

... for a project I am working on and would like to fully support all CM features (themes/ addons/ etc) either by extending the existing plugin, or creating a new one.

I think it is impossible for one man, to support all cm features. ... Codemirror is a toolbox, you can use to create your own IDE. see: the brackets.io project. There is a big company that creates it and a community that creates plugins for brackets ;)

So what we need, is a codemirror editon, that works for our needs. Users can use them or drag and drop plugins from this edition. So you can create your own custom edition.

Or we use: http://twguides.org/customizer.html which means you need well defined functionalities, that are packaged together.

eg:
keymaps:
 - vim
 - sublime
 - ...

theme: A, B, C .. imo only one is needed.

highlighting:
 - tiddlywiki (TBD)
 - javascript
 - CSS
 - html
 - mixed mode
 - text/plain
 - .... + 81 more languages :)

Misc:
 - search and replace
 - autocomplete for
   - tiddler names
   - tags
   - macros and widgets
 - inline editing of transcluded tiddlers :)

--------------

Codemirror needs to be updated to V 4.6 ... At the moment TW uses 4.1
There needs to be an automated process to update code from codemirror.net. .. They are moving with an incredible pace. So if you need to update plugins from there, imo it has to be done automatically. Otherwise you are doomed.

---------------
@TheDiveO

Is anyone thinking about adding a TW5 syntax highlighting mode to codemirror in a similar way codemirror supports a TWC mode?

I was thinking about to implement it, to get rid of the highlight plugin, which uses different colors.
But at that time the parsing mechanism was still in flux and I didn't want to change it every 2 weeks :)

Then I started using the brackets IDE, so there isn't much need for me, to use TW as a development environment atm. What would be interesting though, is a new TiddlyWiki syntax highlighter with the new wikitext syntax.

The only problem is, that a day has only 24 hours and TW software doesn't pay the bills :)

just my 2 cents
mario

[Ed Dixon]

@PMario Thank you for this explanation and much respect for originally doing this in TWc as well as  everything else you do here. The community here is just amazing! You have given me very good advice and much to consider here.

Thanks,

[Danielo Rodríguez]

The codemirror plugin was initially created by Jeremy, as a proof of concept. Then someone hardcoded several cm keyboard bindings (cm plugins) into the TW cm-plugin, to make it more useful: emacs, vim, sublime. The result is pure bloat. ... CM keyboard bindings are plugins for a reason. ... nobody needs them all. 

I think you are talking about me here. 

Maybe we can start discussing how Codemirror plugin should work.

•  We need a control panel. In that control panel we want to 
  
• Define which kind of tiddlers codemirror should edit.
• Define which theme do we want
• Define what keybinding do we want
• We need a way the control panel can detect installed plugins
• How should be a plugin structure be? Same naming as the original codemirro?

Lot of things to discuss here.

[PMario]

On Saturday, October 4, 2014 7:02:31 PM UTC+2, Danielo Rodríguez wrote:

The codemirror plugin was initially created by Jeremy, as a proof of concept. Then someone hardcoded several cm keyboard bindings (cm plugins) into the TW cm-plugin, to make it more useful: emacs, vim, sublime. The result is pure bloat. ... CM keyboard bindings are plugins for a reason. ... nobody needs them all. 

I think you are talking about me here. 

no. I just did a summary out of my memory.

 

Maybe we can start discussing how Codemirror plugin should work.

•  We need a control panel. In that control panel we want to 
  
• Define which kind of tiddlers codemirror should edit.

There is a edit type mapping already: https://github.com/Jermolene/TiddlyWiki5/blob/master/plugins/tiddlywiki/codemirror/EditorTypeMappings.multids

• Define which theme do we want

No theme. The cm default settings are fine. Other themes are plugins. Same with additional language modes.

• Define what keybinding do we want

Default cm keyboard handling is fine for most users. Every other key bindings are plugins.

• We need a way the control panel can detect installed plugins 

imo the $:/ControlPanel already detects plugins. TW defines a structure, how plugins are packaged. They can have there own readme + docs
Active plugins can add there own tabs to the ControlPanel. So the user can adjust settings. All mechanisms are part of the core.

• How should be a plugin structure be? Same naming as the original codemirro?

If it's an addon or keymap or mode in cm, it is a plugin for TW.
I think we should name the plugins exactly in the same way as cm does. So there should be less confusion.

-mario

[TheDiveO]

While I agree on a modular codemirror distribution based on TW5's incredibly useful plugin mechanism, there is one thing missing at the moment: the dreaded config tiddler. I would like to suggest that we modify the existing codemirror wrapper to iterate over all module library tiddlers that are also tagged as, say $:/tags/Codemirrorlib. This would allow end users to simply add codemirror plugins to their TW5s without having to wire up all the pieces through the codemirror config tiddler.

What do you think?

As for your naming suggestion ... I would like to see codemirror or at least cm in the name ... otherwise we may see name clashes with other tiddlywiki plugins?

[Ed Dixon]

I do appreciate efforts to make this available. I may not have the understanding of TW architecture to make these types of modifications yet,  but I do clearly understand my need and desired use for this plugin. It is my hope to offer a very flexible student note book that can be used with all grade levels and subjects, providing as many of the tools needed as possible and all self contained for the student in the book itself. Having taught for many decades and in many different learning environments (K-college), I have discovered that a tool like this will only be utilized if it meets all the basic student / teacher needs and depending on subject much of the required software may not be as available to the student as we would like both in and out of the classroom making even a local server with Learning Management System and TW Iframes ineffective. By providing portable equivalents to the most common tools needed to make notes for as many use cases as possible and integrating those with the existing tagging and note taking tools we can give something of real utility to the students. Due to TW5's common format, single file design, and with the awesome work of several here (TW for Scholars and NoteStorm) we can provide much of this needed functionality already for the organization and citation of the student notes and efforts. What I would like to see is added tools to allow the student to add these notes based on the discipline under study and where native language text may not be the only language used. For example for math we have Ka-Lite which is awesome and provides exactly what I am talking about for several disciplines, for computer classes we could have CodeMirror with these issues addressed but in order to do so we need to support features that may not be common to the basic TW main editor purpose and this type of "config tiddler" to support plugins within the plugin in a modular plugin design (if I am understanding that goal correctly) to add themes and programming languages, and features such as folding, I agree would be preferred.

On the wish list: I have seen some work on Music Notation (a subject near and dear to me) but no standout ready as of yet that I am aware of. The addition of d3.org and current discussions with working with SVG elements in TW5 brings hope that a mind map based on tagging or fields is in the not too distant future. Providing multi-language support in the editor as a drop-down choice would also be a useful tool where the student could choose a language other than native to support foreign language courses, and support for a greater number of external file types where the student could save a copy of linked files to a TWStore folder that could accompany the TW5 file on a thumb-drive providing temporary storage of specific course resources (like a weekly video assignment, or book in pdf format, etc) would in my opinion pretty much wrap this up and we are already so close. A lot of this functionality is already available and I am sure others have developed work arounds to include useful items I haven't even thought of here. But I think by now you get the idea.

Some concerns I have specific to this discussion (and please keep in mind I am primarily a computer use trainer) is the use of modes, themes, and language auto completion features within the CM editor. Having CM as the main editor is not as important (in my opinion) as having these features supported (perhaps as a second editor?) The reason Modes are important to me is that I have taught classes in using VIM and EMACS and having a VIM native language editor in the browser would have been a dream come true for me in teaching those classes. As it was, the students had to reboot the windows machines (without gvim installation rights) using live cds then drop to shell just to get to a vim editor and was the greatest hinderance in effectively showing the best features vim provides (same true of EMACS). Having taught computer classes for a very long time I have discovered that the student gains much more confidence and will use the program much more where personalization or customization features (such as the CM themes) are available and take much pride in setting up "their" system which in this case is the editor itself and will result in the student using it much more. The other features like code folding, autocompletion, etc make the experience much more like working in a full IDE and could be just enough to provide needed experience with those concepts so that a full IDE environment might not be necessary depending on course. So to me it is not about having those features because they are there but because of the utility provided to both teacher and student in having them but there is a real need to control the setup of the editor for each use case. Not to mention the functionality all of this provides to a real designer / developer using these tools both in TiddlyWiki and for other projects.

Of greater concern to me and where I am at hung right now with my understanding of TW5 architecture and adding resources in general is why is this is so complicated when even the flat files from the CM project provide all of this without even running in a server. The files in the demo folder of the project will allow any of these specific features to be tried and tested directly and they seem to have gone out of there way to make it play as nice as possible in any given deployment. I feel this must have something to do with conflicts in DOM manipulation between TW5 and plugins like this but it is my understanding there are ways to work around some of these issues such as require.js and its use of Asynchronous Module Definitions when used with JQuery etc... Please continue to be patient, I can see I am missing a big part of the picture and I am working on it. However, at this point, I do wonder if there might be a way to create a wrapper to allow any javascript plugins use in a specific tiddler while hooking into the TW5 mechanisms to access generated tiddlers, and provide regular tagging, and all of the other features native to TW5.. I really do appreciate the community here and all efforts to bring this type of functionality to TW5. 

Thanks,   

  

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/JG7bDM-eMeU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[Danielo Rodríguez]

El domingo, 5 de octubre de 2014 01:06:09 UTC+2, PMario escribió:

There is a edit type mapping already: https://github.com/Jermolene/TiddlyWiki5/blob/master/plugins/tiddlywiki/codemirror/EditorTypeMappings.multids

I know about the mapping. I'm who proposed it and I followed it very close. But the tiddler you are mention is just an "info" tiddler. There is no way to edit that values from there in an easy way. I would like to see some drop down menus to do this. 

• Define which theme do we want

No theme. The cm default settings are fine. Other themes are plugins. Same with additional language modes.

Maybe fine for you. I'm talking about the possibility to SWITCH themes. I don't want to uninstall or disable a plugin just to select a new theme. I would like to do from a drop-down in an easy,fast and logical way.

 

Default cm keyboard handling is fine for most users. Every other key bindings are plugins.

Again, I'm talking about a single control panel to manage EVERYTHING related to Codemirror. Of course default keyboard is fine, but as I said, I would like to have the ability to change them easily.

 

• We need a way the control panel can detect installed plugins 

imo the $:/ControlPanel already detects plugins. TW defines a structure, how plugins are packaged. They can have there own readme + docs
Active plugins can add there own tabs to the ControlPanel. So the user can adjust settings. All mechanisms are part of the core.

The same misunderstood again. I want to manage all the codemirror configurations in a single place. So maybe the codemirror control panel should load configuration tabs based on tags like the system control panel does. 

• How should be a plugin structure be? Same naming as the original codemirro?

If it's an addon or keymap or mode in cm, it is a plugin for TW.
I think we should name the plugins exactly in the same way as cm does. So there should be less confusion.

-mario

I agree with the fact that the naming should be as similar to the codemirror plugin as possible.  But we can not name it as /core/ because it is not a core plugin.

[Danielo Rodríguez]

El domingo, 5 de octubre de 2014 21:41:56 UTC+2, TheDiveO escribió:

While I agree on a modular codemirror distribution based on TW5's incredibly useful plugin mechanism, there is one thing missing at the moment: the dreaded config tiddler. I would like to suggest that we modify the existing codemirror wrapper to iterate over all module library tiddlers that are also tagged as, say $:/tags/Codemirrorlib. This would allow end users to simply add codemirror plugins to their TW5s without having to wire up all the pieces through the codemirror config tiddler.

What do you think?

 I agree with you (and I said this several times) that the current way to configure codemirror is everything but intuitive. But, I don't think we need the plugin to iterate over all library tiddlers. The only thing we need is an easy way to modify the config tiddler. That's the control panel I was talking about.

 

As for your naming suggestion ... I would like to see codemirror or at least cm in the name ... otherwise we may see name clashes with other tiddlywiki plugins?

As I said to Mario, I agree they should be very similar, but we can not include /core in the name 

[TheDiveO]

On Monday, October 6, 2014 10:17:05 AM UTC+2, Danielo Rodríguez wrote:

El domingo, 5 de octubre de 2014 21:41:56 UTC+2, TheDiveO escribió:

While I agree on a modular codemirror distribution based on TW5's incredibly useful plugin mechanism, there is one thing missing at the moment: the dreaded config tiddler. I would like to suggest that we modify the existing codemirror wrapper to iterate over all module library tiddlers that are also tagged as, say $:/tags/Codemirrorlib. This would allow end users to simply add codemirror plugins to their TW5s without having to wire up all the pieces through the codemirror config tiddler.

What do you think?

 I agree with you (and I said this several times) that the current way to configure codemirror is everything but intuitive. But, I don't think we need the plugin to iterate over all library tiddlers. The only thing we need is an easy way to modify the config tiddler. That's the control panel I was talking about.

I agree on the control panel -- and for this to work we need some infrastructure to properly detect codemirror-specific extensions. One way to detect the individual library tiddlers would be using well-known title path prefixes. The downside here is that when later moving the plugin to a different namespace, things will get ugly. Also, this make packaging individual add-on plugins from the same single namespace unnecessarily complex and fragile.

My idea would be to have

• $:/plugins/TheDiveO/codemirror -- the base plugin with support for generic highlighting, javascript, css, and (later) TW5 simple markup.
  
• $:/plugins/TheDiveO/emacs-binding -- additional keybinding
• $:/plugins/TheDiveO/eclipse-theme -- ...
  

 

 

As for your naming suggestion ... I would like to see codemirror or at least cm in the name ... otherwise we may see name clashes with other tiddlywiki plugins?

As I said to Mario, I agree they should be very similar, but we can not include /core in the name 

Then I will probably do my own playground trial as $:/plugins/TheDiveO/CodeMirror until we see clearer as to which way the tiddlywiki/codemirror plugin may head in the future.

[Stephen Kimmel]

For me, we would like to keep a CodeMirror plugin useful to the maximum number of people with very simple additions to allow the specialists to enhance it to meet their needs. There are or should be a lot more Tiddlywiki users than Tiddlywiki programmers. What I need is an enhanced text editor and enough accessible documentation that I can understand how to make the thing work. My wishlist includes:

1. bracket matching that includes the Tiddlywiki formatting codes. I've spent more time than I can count trying to find the mismatched ` that is matched by a '.
2. search and replace.
3. the basic keysnippets functionality of adding a Tiddlywiki format code to the beginning and end of a selected passage.

Many of the CodeMirror functions, hiding a block, moving a block, transposing letters, are nice to haves. While I can see why folks would like themes and autospelling and such, I wouldn't want to try to remove them manually.

I like the idea of a control panel, perhaps like the old TWC one where you could simply check off the options you want. If it is going to result in even more code bloat than Tiddlywiki already has, then I don't think we need to have everything pre-installed.

The config tiddler was a hassle. I would have liked to have had one prebuilt as part of the plugin. I don't know if it would work but could we include the necessary lines for the available modules and comment them out?

[Ed Dixon]

Hi Stephen,

I agree and am all about removing everything that reduces resources in everything that I do. (You probably can't tell that from the lengths of my posts above LOL but I am). A minimalist to the extreme here which is probably why I find TiddlyWiki so fascinating. I do think though as the code mirror plugin is an optional plugin to begin with that there should not be anything restricting another plugin from providing the full set of Code Mirror features with a configuration control panel for it if that is possible, and if there is a need for it. That said I am with you, if I did not have a specific need for it, I would not even be using the current one as the basic editor seemed fine for that purpose.

I know I must be missing something here but to me seeing how the CM package is complete as a standalone javascript plugin, is replacing the main editor the only way to add CM capabilities to the TW5 file?  I feel once I fully understand why spawning another editor instance or wrapping the functionality and hooking it to TW5 capabilities is not possible, I will have a much clearer picture of what is possible here but my gut is telling me this could be done and might open worlds of possibilities for much easier integration of other useful scripts as well.

Thanks,

--

[Stephen Kimmel]

I know I must be missing something here but to me seeing how the CM package is complete as a standalone javascript plugin, is replacing the main editor the only way to add CM capabilities to the TW5 file?  I feel once I fully understand why spawning another editor instance or wrapping the functionality and hooking it to TW5 capabilities is not possible, I will have a much clearer picture of what is possible here but my gut is telling me this could be done and might open worlds of possibilities for much easier integration of other useful scripts as well.

Eric's QuickEdit package for TWC demonstrates that the editor can be modified to achieve the functionality we would like. Someday, there will be a package like that for TW5. Danielo's Keysnippets also expands on the existing editor and adds a lot of functionality as well. So clearly a lot of this can be done. It just hasn't been done yet. CM is probably overkill but... if I could get CM and Keysnippets working together I would be much closer to where I want to be. Ultimately something like the QuickEdit package is probably the right answer.