JavaScript limitations & some possible solutions

[Chris H]

I'm a long-time Memento user, but didn't bother with scripting until now.  I've been reading the docs, to see if it's possible to get it to do what I want, and noticed that scripting has quite a few limitations... 
But after a lot more thinking (and searching), I realised there are work-arounds for at least some of them, which I'll list below.  Please let me know if I'm mistaken, or if there's a better solution:

* No way for a Script field to access the fields of other entries (as libs() is not available).  However, this is not quite true - there is a limited way to access another entry, and you can use this to access all the entries with a bit of work:
If your entry has a field which Links to another entry, then you can access the fields of that other entry...  So if you give each entry a link to the Next (and/or Previous) entry in the library, then you can step through every entry in the library!  So all you need now is a Script (ran manually or otherwise) which goes through all entries, and links them together using the Next (and/or Previous) fields.  This solution is tested as working :)

* No way to store "global" state.  The solution is to have another library, and have each entry link to an entry in that other library.  In simple cases that other library would only have one entry, but in more complex cases you can have several entries (which store "semi-global" state).  So you then just need a Script which runs when an entry is created, and have it set the entry to link to the "global" state. 
However, as Memento appears to store when an entry is linked to by another entry, this will lead to one entry knowing it's linked to by hundreds of other entries.  I'm unclear if this will cause Memento to perform badly (hopefully not).  But at the very least you should tell Memento to NOT "Display on the card" & "Display aggregation" all the entries which link to an entry.

* No way to access or control the Filters.  The solution I thought of is that every entry would have a boolean field (let's call it Hidden), and a single filter which hides entries with Hidden=True.  Then instead of users changing filters in the normal way, you would have an Action script button(s) for changing what is filtered.  Your script would probably have Arguments that the user can adjust, to choose what they want filtered.  The script then goes through every entry, setting Hidden accordingly.  I suspect this will be a bit slow, but seems like the best option unless JavaScript is allowed to control Filters directly.  It could potentially also be optimised, by only recalculating Hidden for entries that may change state (although this would be a bit tricky).

Another alternative would be to make the Hidden field be a Script itself (with "Execute script in real time" enabled), which would calculate whether the entry should be visible (based only upon the entry's own fields - and those of any entries that it links to).    You can use my 'global state' solution (i.e. a link to a shared entry in another library) as a way to give users control over what is filtered (and have that be persistent).  On the downside I suspect this will perform worse.

* Limited ability to store data, as all entries must have the same fields.  Probably obvious, but that just means you need to have multiple libraries!  And probably Group them together to avoid things getting cluttered.
Effectively you can imagine that each Library is a class in an object-orientated language, and each Entry is an object of that class.  So if your big program needs several types of objects, then you just need to have several libraries!

* libs().entries() does not give you any control over the order of the array, so you have to sort it every time (which is a bit slow).  I read that the order of entries in the array is in Creation Time order.  It also appears that you can read & write the "creationTime" field of an entry, so I'm planning to set this field directly.  As long as I only want the entries() array in one particular order, then I should be able to force Memento to return the entries() array in that order!

[Chris H]

I just had another idea (which was kinda obvious after all the above) :

* No way to access or control the Sort order.  The solution is to directly change the built-in "creationTime" field (or else add a new Integer field for that purpose), and always have Memento set to sort by "Creation time" (or that Integer field). 
Then you 'only' need to change the value of that field, for every entry, and that will change the Sort order.

[Bill Crews]

Before we get started, there are 2 categories of JavaScript in Memento -- JavaScript field type and other scripted things, like triggers, actions, and custom data sources. You seem to be more familiar with JavaScript fields. The JavaScript capabilities in JavaScript fields are intentionally limited, so as to keep them simple, for non-programmers. The idea is to enter an expression, not really a full-fledged script.

Triggers, actions, and cushion data sources can use the Memento JavaScript Library (see that wiki page), while JavaScript fields cannot. To keep it simple, I'll just refer to actions here, but the same is true for triggers and custom data sources.

* No way for a Script field to access the fields of other entries (as libs() is not available).

It is true that JavaScript fields are limited as you say. But actions can use entries() to discover the entries of any library. However, as you say, even in actions, there is no libs() function to use to discover all the libraries in a group or in your entire database.

However, using libByName(), you can access the entries of any library you can name, so long as you have the permission to do so.

* No way to store "global" state.

I didn't follow all this, but normally (with actions), you don't have to do all this. If you're just trying to get access to entries of another library, you can do that with libByName(). Save linking for when you really need to link specific entries to specific other entries because your solution calls for it, not as a constructed mechanism to access entries in general.

By the way, in actions or JavaScript fields, when you access a link, you always get back an array of entries. If it's a one-to-many link, there will be only one entry in the array, but it's an array nonetheless, which is why you need that [0] thing that may seem extra at first.

* No way to access or control the Filters.

What you're not telling us is what you're trying to accomplish. If you really need a filter, use a filter. If you're trying to accomplish something else, let us in on your secret, and maybe we can help you accomplish it.

* Limited ability to store data, as all entries must have the same fields.

Yeah, maybe, but it depends on what your trying to accomplish. Normally, you don't have to do all this. If you're trying to turn Memento into an object-oriented programming language with an object data store, good luck with that. Get Python with SQLite or something.

* libs().entries() does not give you any control over the order of the array, so you have to sort it every time (which is a bit slow).

Your idea will probably work.

[Bill Crews]

Sorting isn't always too slow for all solutions, so use this technique when you need to, but when you don't, you might want to just keep it simple.

[Chris H]

Hi

Bill Crews wrote:

Before we get started, there are 2 categories of JavaScript in Memento -- JavaScript field type and other scripted things, like triggers, actions, and custom data sources. You seem to be more familiar with JavaScript fields.

No, I investigated both, since (as you say) Script fields are rather limiting.  However Script fields also allow something that Triggered scripts don't: They can be automatically run on all items, while Memento is filtering/sorting items for display (which means that can potentially be more up-to-date than fields that are updated by Triggered scripts, and are certainly easier to keep up-to-date - but at the expense of being evaluated every time).

 

* No way to store "global" state.

I didn't follow all this

Sorry, this was meant to be taken in the context of Script fields, and I should have called it "shared global state". 

Script fields can only access data stored in the current field. This is a BIG limitation if you want to use a Script field to do something that can be controlled by an Action script.

e.g. You have a Script field called Hidden, and you want to use that to control whether an entry is filtered or not.  Ideally you would have some shared global variable, but this is not supported (so hacks are necessary - such as every entry linking to the same entry in another library).

 

* No way to access or control the Filters.

What you're not telling us is what you're trying to accomplish. If you really need a filter, use a filter. If you're trying to accomplish something else, let us in on your secret, and maybe we can help you accomplish it.

I want to have lots of entries, but allow the user to filter them based on a specific field.  When the user performs an Action on that entry, the script needs to know which filter is active, so that it can make appropriate changes in another library.  (I could explain the specific details, but it probably wouldn't make any more sense to you than this general description.)

Since Memento does not allow Scripts to see which Filter is active (nor control which Filter is active), the only solution I can think of is create my own filter system that the user can control with an Action button.  And since shared global state is not supported, the Action script would store the current 'filter' settings in the entry of another library.  If I chose to make the "Hidden" field a Script, then I would need every entry to link to the (shared) entry of that other library.  Yes, this is getting a bit complicated :-)

* Limited ability to store data, as all entries must have the same fields.

Yeah, maybe, but it depends on what your trying to accomplish. Normally, you don't have to do all this. If you're trying to turn Memento into an object-oriented programming language with an object data store, good luck with that. Get Python with SQLite or something.

I've used Pydroid 3 a bit on my Android phone, but implementing a decent user interface seems like a lot of work (if possibly at all), where-as Memento has already done the hard work...  So I'm investigating if I can get Memento to do what I want!

[Bill Crews]

OK, I'm not sure I can help much, then, but the following comes to mind...

Sorting, grouping, and filtration (and charting and aggregation, too) in Memento are all considered to be display-related features, displaying existing data from the database. Actions and script fields are in the data content realm, and once that data is resolved, then the display-realm stuff can work to display it the way you want.

So, getting content to be based on display specs is not really meaningful. If your content is being shown and you change what filter is being displayed, there is no communication about it to base anything on. Supposedly, you're just changing how you want the data to be displayed, not the data itself.

Beyond that, you're on your own. Maybe you can develop something useful to you. My guess is that doing something that radical will cause you to run into problems, such as performance issues or other issues that don't normally show up for Memento users. Performance issues exist even when using Memento normally. But good luck!