How about integrating dictionaries with J primitives?
Will Springs
I’ve been following the discussion on the new dictionary add-on with great interest.
I would like to echo the suggestions to integrate dictionaries with J primitives, which makes J’s concise and expressive syntax stand out.
The goal would be to allow users to use keys wherever numeric indices are currently used, benefiting from the performance and thread optimisations of dictionaries discussed in other threads. Keys could also be used in a multidimensional fashion, staying true to J’s array paradigm (e.g., fetching sales amounts from a region for a particular product from a table). Ideally, every primitive could have a specific meaning for dictionaries.
Here’s a rudimentary starting point of how that could work (including some ideas from other posts):
| Construction or activation of dictionary optimisations on an existing array | $ (shape) with _. (indeterminate) For example, to create an empty boxed dictionary on a table with two columns, keys enabled on axis 0 MyDict =. _. 2 $ a: |
| Insert or update | } (amend) |
| Get | { (from) |
| Has | e. (member) |
| Delete | -. (less) |
| Count | # |
This is meant for illustration only and I understand this will lead to lots of work and discussion to achieve consistency, and unfortunately I won’t be able to contribute as much as I’d like for various reasons.
For instance, I’m aware the tree/hash option from the add-on isn’t resolved with this notation, and if we want to enumerate all keys/values from a dictionary or use primitives like (Append) or ; (Stitch), we need to define how to represent the values and keys in the same array altogether. In fact, the add-on provides already an elegant solution to this, and also allows for multidimensional or hierarchical (boxed) keys. And maybe the use of special combinations is the more pragmatic approach to what I am seeking for.
I’m concerned with the risk of proliferating optimisations and objects for data structures coming from other languages paradigms (dictionaries, lists, sets, queues, quaternions...), losing the initial expressivity of J. Arguably in J, such structures could stay transparent to the user, and the interpreter should apply the optimisations without the user even noticing.
Most of all, thanks for the great work on kv and the new add-on. Sorry if this post sounds too simple or if I missed important reasons for not integrating the primitives with dictionaries. I’d be interested to learn more about that.
Pascal Jasmin
To tweak your proposal a bit,
A dictionary APPEARING to the rest of J as being 2 columns (keys ,&< values) of equal count, that have a defined order, is an ideal interface.
get being a separate cover function for foreign is fine, because its an operation spread accross the 2 columns. It could still work on any 2 columns (even if they are not an official dictionary) as an interface that passes strings/boxes to { equivalent (get) in one column, to retrieve from 2nd column. Similarly a has function that is optimized on dictionaries should still do the same for 2 columns of equal count.
It is fine for put/del to be the only supported interfaces for modifying a dictionary, but I'd strongly prefer if those functions returned the modified dictionary instead of EMPTY. Like }, the in place optimization would be done by looking if same name as y is assigned back. put/del could also work on the plain J 2 columns.
the extra range functions from tree type are useful, but a slow (or using inserted order) implementation for plain J 2 columns (or hashed type dict) would be nice.
_________
I am ambivalent towards a dictionary containing extra undisplayable columns that must never be touched, but J should not crash if:
The dictionary is boxed, with usual purpose of storing it in another dictionary, or at a deeper level within same dictionary.
A dictionary can be erased or if it is allocated on the stack, go out of scope and do any required housekeeping before freeing up memory internally.
_________
A bit like JP's function, the create function would take 2 columns as y (can be empty) with optional x the type shape restrictions for k/v. distinct create functions for tree/hash can make sense, or create is an adverb that takes 'tree' or 'hash' as m. When x is not provided, dictionary is locked to type/shape of the 2 columns.
________
some nice to haves,
the tree and hash type dictionaries could have both optimizations applied simultaneously?
while there is no obvious need for hash values as an accessible column, the linked list system of the tree structure could be exposed because ordering doesn't have to be limited to lexical sorted order. An insert function would let you provide an intended sorting order, or you could process the dictionary to replace any "sorting order".
the difference between a dictionary and a table is extra value columns. Stored as an inverted table with list of keys in first column, and field values as rest of columns. 1 value per column for each key. The above create function if y has more than 2 columns would make a table. get would return inverted table with values corresponding to each requested key, put would require a similar inverted table format for multiple keys. This provides much better storage efficiency than a list of boxed values as the value for each key.
______________
Benefits are that new functions work on J columns, and J primitives work on the APPEARANCE of dictionaries/tables as a list of columns. There is no need for Henry to provide reverse value lookup, and the user can still "freeze" a dictionary/column list with ((columns or dict/table) {::~ (value column)&i:) to speedup reverse lookup on a snapshot.
On Thursday, February 26, 2026 at 04:01:15 a.m. EST, Will Springs <will.g....@gmail.com> wrote:
I’ve been following the discussion on the new dictionary add-on with great interest.
I would like to echo the suggestions to integrate dictionaries with J primitives, which makes J’s concise and expressive syntax stand out.
The goal would be to allow users to use keys wherever numeric indices are currently used, benefiting from the performance and thread optimisations of dictionaries discussed in other threads. Keys could also be used in a multidimensional fashion, staying true to J’s array paradigm (e.g., fetching sales amounts from a region for a particular product from a table). Ideally, every primitive could have a specific meaning for dictionaries.
Here’s a rudimentary starting point of how that could work (including some ideas from other posts):
Get { (from)
Has e. (member)
Delete -. (less)
Count #
This is meant for illustration only and I understand this will lead to lots of work and discussion to achieve consistency, and unfortunately I won’t be able to contribute as much as I’d like for various reasons.
For instance, I’m aware the tree/hash option from the add-on isn’t resolved with this notation, and if we want to enumerate all keys/values from a dictionary or use primitives like (Append) or ; (Stitch), we need to define how to represent the values and keys in the same array altogether. In fact, the add-on provides already an elegant solution to this, and also allows for multidimensional or hierarchical (boxed) keys. And maybe the use of special combinations is the more pragmatic approach to what I am seeking for.
I’m concerned with the risk of proliferating optimisations and objects for data structures coming from other languages paradigms (dictionaries, lists, sets, queues, quaternions...), losing the initial expressivity of J. Arguably in J, such structures could stay transparent to the user, and the interpreter should apply the optimisations without the user even noticing.
Most of all, thanks for the great work on kv and the new add-on. Sorry if this post sounds too simple or if I missed important reasons for not integrating the primitives with dictionaries. I’d be interested to learn more about that.
Henry Rich
To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.
Will Springs
I hope it will still possible to use J primitives as far as sensible to provide a seamless experience to the user.