j9.7.0-beta12 available

[Eric Iverson]

If you have already installed 9.7, the upgrade is easy:
   load 'pacman'
   'upgrade' jpkg 'jengine'

To install 9.7 beta:
 https://code.jsoftware.com/wiki/System/Installation/J9.7

Release notes:
 https://code.jsoftware.com/wiki/System/ReleaseNotes/J9.7

Questions/bug reports/etc. should go to the beta forum.

[Marcin Żołek]

Beta-12 introduces range query support for Dictionaries. To upgrade the addon:

   load 'pacman'

   'update' jpkg ''

   'upgrade' jpkg 'data/dictionary'

Documentation now includes an implementation of the well-known shortest paths Dijkstra's algorithm using a tree dictionary.

Marcin

[Pascal Jasmin]

thank you very much for mget.

I'd suggest also:


getall_jdictionary_ =: since__en2lat@a:  NB. called as nilad ''
display_jdictionary_ =: (,. L: _1)@:getall_jdictionary_

perhaps range '' could return the full dictionary (getall).


NB. Create the map, which remains as dict. dict is marked nondisplayable because 1 {:: dict is.
NB. If 1 {:: dict (keys) is an indirect type, it is death to touch or display any part of 1 {:: dict that is on the empty list.
NB. It might be better not to assign dict, to make it impossible to access 1 {:: dict from console level. But we have to be able to run 16!:_5 on it - make 16!:_5 an adverb


L. dict__en2lat is also death.

but < and }. is fine, even if data is not displayed inside.

with support for < and

lr_z_ =: 3 : '5!:5 < ''y'''

lr }. dict__en2lat

(<undisplayable),(<undisplayable),(<0$00),(<0$00),(<),<


This suggests that a functional approach to dictionaries, with nesting, is possible (with <), and there is enough information for an isdictionary function with lr though I have no idea how consistent the values for 3 }. dict are.  It is not clear whether resize_jdictionary_ is needed for automatic resizing as needed.

discouragements on should I even if I can I are welcome.

other questions:

for hash type, are symbol keys unnecessary compared to boxed strings, because the strings will be hashed?

My impression is that keys are stored in sorted order anyway to support the fancy query methods.  Does the tree type avoid the "physical sorting" to hold pointers instead?  Wiki says "optimized for range queries", but is hash type just sorted each time, to still support them?

(1) 16!:_5 dict will physically "pack" (completely remove deleted items), but in a functional environment where no more reference to a variable/dict exists, there could be a crash if a "kill" function is not called?

thank you.

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Henry Rich]

We will support load and unload for accessing all the kvs, next beta.

There is nothing useful you can do by accessing the dict noun directly. If you want to have a dictionary contain a dictionary, store the locale number. Foreigns with negative n are not documented or supported and you shouldn't use them. 

  

I don't understand the kill question. You should consider the dictionary type as opaque and access it only through the provided functions. 

We consider the dictionary addon to be an improvement on the symbol datatype. Symbols will be removed from J in the next release.

Keys are never explicitly sorted. Hash maps are unordered. Tree maps keep a balanced sort tree for indexing the keys.

Henry Rich

[Pascal Jasmin]

> We will support load and unload for accessing all the kvs, next beta.

the following only works on tree type.

getall_jdictionary_ =: range@:(min ,&>&{. max)

since__ a:  will not work on numeric boxed keys though before__ a: returns boxed numeric

does load/unload mean similar to above serialization but including hash type support? 

>  Symbols will be removed from J in the next release.

Very disappointed initial reaction.  Keyed access can exist without sorted (self lexical)/unique keys.  ordered by different or natural insertion order means.  Many to many table relationships as example.  There seems to be massive functionality not intended to be provided by dictionary class/foreigns that can be with symbols.  Perhaps symbols could be used with tree type to improve direct access in addition to "sorted appearance functions"

> Keys are never explicitly sorted. Hash maps are unordered. Tree maps keep a balanced sort tree for indexing the keys.

put__en2lat~ ;/0 2 3 4 5 6 1  NB. tree boxed boxed dict type

both getall__ and before__ return sorted order despite inserted in different order.

perhaps with 'hash' type, the above functions could still be defined, but return keys in insertion order?

> I don't understand the kill question. You should consider the dictionary type as opaque and access it only through the provided functions. 

inside jdictionary class, the dict variable holds the state even if undisplayable/don't access the boxes.  This still allows for "functional access" as opposed to OOP access.  The only obstacles are:
1. whether the resize function definition is not needed for automatic resizing (which I assume takes place). 
2. "(1) 16!:_5 dict NB. clear the empty chain in the keys to avoid errors freeing it" if a temporary dict (direct without class) was created, is this extra cleanup only needed if items were deleted for memory (from temp dict) to be freed when function exits?  If I don't delete any items, I can erase 'dict__' without error.  (later access crashes of course).  The "kill question" applies to a non-OOP use/implementation of 16!: functions, and under what circumstances would  (1) 16!:_5 need to be called, by a kill function (where just variable falling out of scope failed to release all memory).  An obvious follow up request, can whatever (1) 16!:_5 does, be an internal check instead of requiring it as part of destroy function?

OOP is very flaky in J.  scope of modifiers has changed u./v. may help, but I'm sure confusion/frustration will apply.  Operations on a list of dictionaries/objects (keeping multiple references in a list a common/useful organization... and this is J) is messy.  Best I remember is:

inl_z_ =: (cocurrent@] ".@] [)"1 0 

that requires building monad or dyad expressions as strings, with any modifiers not in z as fully qualified path (so just put all adverbs you might use in z).  Alternatives still require lr/ar building.

the memory management/creation/destroy and long typing of OOP code in addition to losing array functionality made me learn to abandon it entirely.

I'm happy to build and share a functional version. I hope symbols stay, that auto-resize happens internally, and (1) 16!:_5 is clarified.

[Henry Rich]

load/unload read/write all keys, with an option for indicating that keys
are in ascending order (which greatly speeds up loading a large tree).

Henry Rich

[Marcin Żołek]

Hi Pascal,
If you plan to use multiple dictionaries that need to be indexed as elements of an array, at least two approaches are possible:

1. Array of dictionaries.

   load 'dictionary'
   ]mydicts =: {{'hash' conew 'jdictionary'}}"0@:i. 7  NB. Create array of 7 dictionaries.
┌─┬─┬─┬─┬─┬─┬─┐
│0│1│2│3│4│5│6│
└─┴─┴─┴─┴─┴─┴─┘
   ]d =: 3 { mydicts  NB. Select one of dictionaries.
┌─┐
│3│
└─┘
   (put__d~ 100&+) i. 3
   get__d i. 3
100 101 102

2. Use one dictionary instead of an array, encoding the array index directly in the key.

   ]mydict =: 'jdictionary' conew~ 'hash' ,&< 'keyshape' ; 2  NB. Keys are (index, true key).
┌─┐
│7│
└─┘
   (100&+ put__mydict 3&,.) i. 3
   get__mydict 3 ,. i. 3
100 101 102

While the first approach may be limiting when you want to put keys and values to multiple dictionaries within a single J sentence, the second approach does not seem to have this limitation, because array of dictionaries is simulated by one dictionary with expanded keys. Also, the second approach may be beneficial for range queries, because everything is stored in a single dictionary. For range queries, create the dictionary as 'tree' instead of 'hash'.

Is any of these approaches suitable for your planned use case of dictionary?

Marcin

[Pascal Jasmin]

A final rant of how my love of J is based on hatred of OOP:  dictionaries are a way to package a bunch of related properties without using classes.  Lua used this approach, and JSON is the same.  It would be ironic to force OOP to access dictionaries.

That said, it is fairly easy to put functional wrappers around OOP access, and within wrapper module avoid all scope confusion from modifiers or other array limitations.

put =: ] [ (4 : ' put__y&>/x '"1 0)
get =: 4 : 'get__y x'"1 0

]mydicts =: {{'tree' conew 'jdictionary'}}"0@:i. 7  NB. Create array of 7 dictionaries.  tree to support getall

mydicts put~ 2&#"1@:,.@:<"1 i.7 3  NB. put 3 keys in each dict

mydicts (get~ ,@{."1) i. 7 3  NB. retrieve first key from each dict

0
3
6
9
12
15
18

d =: 3 { mydicts NB. Select one of dictionaries.

getall_jdictionary_ =: range@:(min ,&>&{. max)

getall__d ''

┌───────┬───────┐

│9 10 11│9 10 11│

└───────┴───────┘

functional approaches allow for more powerful operations without a for loop or multiline selection/assignment of an item then an operation.

in https://github.com/Pascal-J/kv/blob/main/kv.ijs lines 509 on, forkey conjunctions are powerful ways of modifying multiple dictionaries.


I forget why this doesn't work. bug in latest beta?

codestroy"0 mydicts

|rank error in coname, executing monad 18!:5

codestroy d

|rank error in coname, executing monad 18!:5  


the destroy step is a hassle even if functional wrappers are easy.

On Thursday, February 19, 2026 at 05:18:32 p.m. EST, Marcin Żołek <marcin...@students.mimuw.edu.pl> wrote:






Hi Pascal,
If you plan to use multiple dictionaries that need to be indexed as elements of an array, at least two approaches are possible:

1. Array of dictionaries.

   load 'dictionary'

   ]mydicts =: {{'tree' conew 'jdictionary'}}"0@:i. 7  NB. Create array of 7 dictionaries.

┌─┬─┬─┬─┬─┬─┬─┐
│0│1│2│3│4│5│6│
└─┴─┴─┴─┴─┴─┴─┘
   ]d =: 3 { mydicts  NB. Select one of dictionaries.
┌─┐
│3│
└─┘
   (put__d~ 100&+) i. 3
   get__d i. 3
100 101 102


2. Use one dictionary instead of an array, encoding the array index directly in the key.

   ]mydict =: 'jdictionary' conew~ 'hash' ,&< 'keyshape' ; 2  NB. Keys are (index, true key).
┌─┐
│7│
└─┘
   (100&+ put__mydict 3&,.) i. 3
   get__mydict 3 ,. i. 3
100 101 102


While the first approach may be limiting when you want to put keys and values to multiple dictionaries within a single J sentence, the second approach does not seem to have this limitation, because array of dictionaries is simulated by one dictionary with expanded keys. Also, the second approach may be beneficial for range queries, because everything is stored in a single dictionary. For range queries, create the dictionary as 'tree' instead of 'hash'.

Is any of these approaches suitable for your planned use case of dictionary?

Marcin

[Henry Rich]

We added dictionaries to J because they provide something functional
programming cannot: large maps that are very efficient for query and
update, and can be accessed across multiple threads.

codestroy requires an empty argument.  But don't use it!  You will
crash.  Use the functions as documented.

Henry Rich

[Pascal Jasmin]

Thank you Henry for adding useful feature to J.  I beg that you don't remove symbols/useful features outside of your intended scope for dictionaries. 

TIL, the destroy function in an object is not a magic name that gets called by codestroy.  Reference counting language history (that do have the magic) confused me on this.

If I can losely define functional approach as (new)data =. x update data that is independent of in place implementation optimization.  Permits temporary dictionaries for query/filters and later merges with other dictionaries.

It is not a deal breaker to use 16!: functions, or if dictionary is undisplayable, but it is very harsh to rely on user code (addons count) to not crash J by failing to call (documented or not) cleanup function before temporary dict is destroyed by going out of scope.  A nice to have feature would be if last 2 columns of dict (currently 7 boxes) were displayable keys and values.  Perhaps the linked list ordering or user columns (for alternate ordering or extra fields as traditional db table) could be possible, and direct J manipulation of those columns allowed.  There are other means to extend dictionaries, with extra data, and I fully understand if this request is too messy, so I'll just repeat the request to just not crash J if 16!:_5 is not called prior to out of scope.

[Henry Rich]

We decided long ago that the symbols implementation was clumsy and
restricted and that we would remove it when we had a good replacement. 
The Dictionary addon is that replacement.  Current J symbols are a
single system-wide unordered dictionary whose keys are boxed character
strings and whose values are integers.  If you need that functionality
you can recreate it using the addon.

You have locale destruction backwards.  (destroy '') is how you delete
an object, such as a dictionary.  destroy does whatever it needs to do,
and finishes up with (codestroy '').  For some locales you can bypass
the (destroy ''), but a dictionary is not such a locale.  You must use
destroy.

Here's good news: I have just now fixed the implementation so that it
does not expose any noun that would allow a user to crash the system. 
You'll see that in the next beta.

You must not use the 16!:n functions that have negative n.  They are for
Jsoftware use and are subject to change without notice.

It sounds like you want some of your dictionaries to act like regular J
values, such that modifying the dictionary creates a new dictionary with
the changes.  We will support that idea if we together can come up with
suitable functions to do it, but that is a very special and restricted
case of dictionary usage.  The general case, which we want to perform as
well as possible, is a very large dictionary - possibly Gbytes in size -
that is being read and written simultaneously by multiple threads.  In
such a dictionary (1) looking at all the keys or values takes much
space; (2) collecting the keys and values requires freezing the
directory while the copy is assembled; (3) the whole concept of 'the set
of key/values' is murky, since updates are being performed in parallel. 
We want a single get from such a big directory to take less than 40ns,
and we want to provide hundreds of millions of batched gets per second
across all threads even while the dictionary is being updated at maximum
speed.  Creating a table of all the keys would be an expensive and
exotic request in such an environment.

What I think you are asking for is a more gentlemanly dictionary, where
updates are applied one at a time and you can look at the keys or values
at your leisure.  Maybe I can think of it as an offline version of a
general dictionary, one in which updates from other threads are not
possible.

Please think about what features you would need to do what you want.  We
might implement them as functions supported by the dictionary addon. 
But don't bypass the addon!  Any access to a dictionary that doesn't go
through our defined functions will not be supported.  In particular, do
not look at the dict__nnn value, since it is in use in multiple threads.

Functions that I imagine you want are:

keys - return a table of all keys
values - return a table of all values
clone - create a new dictionary that has the same kvs as an old one

Henry Rich

[Pascal Jasmin]

> What I think you are asking for is a more gentlemanly dictionary, where
updates are applied one at a time and you can look at the keys or values
at your leisure.  Maybe I can think of it as an offline version of a
general dictionary, one in which updates from other threads are not
possible.

I'll start here, but rest is more deferential.  No.  while small dictionaries are useful, I believe in bulk operations, and espeically helper modifiers to abstract complex tasks.  I believe functional approach to dictionaries makes these functions easier.  I'll repost https://github.com/Pascal-J/kv as my dictionary implementation.  It is meant for high performance, in its own way, but also convenient syntax.  My current project is to create a generic search/solver function generalized from the complexity of a chess search engine.  Dictionaries are a useful means of organizing all of the board state data, and very large data structures can occur in expanding and evaluating all legal moves as nested dictionaries of same fundamental format.  threading is possible as long as each thread explores a single move, and parent thread takes care of updating its key/value for that move/sub dictionary.  A generalized search/solver function is easier with dictionaries than a flat table where legal moves can be 0 1 or many, and expanding the table must guard against each of these edge cases.  Recursive descent on a dictionary structure is an easy path to generalize to all problems... I hope.

 
> the symbols implementation was clumsy and restricted and that we would remove it when we had a good replacement. 

If I had a dictionary with 100 or 1000+ keys, I'd definitely look at using your implementation.I don't actually know if the overall overhead  of using a dictionary with 10-40 keys with symbols (symbol creation overhead vs get savings) is better performance overall than boxed strings.  Instead of high key volume, my kv implementation, has functions to enhance inverted tables, where a key is a field name.  It seems very approachable to build a RDBMS system this way with tables as higher dictionary and join information part of individual table dictionaries.  But then symbols acting as indexed/hash search in one column to get "record" seems useful, and one key per column makes querying easy to read.

while kv is meant to have any keys/values, practically, general dictionaries have symbol keys, and boxed (mixed) values in kv, through autopromotion.  The kvi function is meant to "freeze" a dictionary for retrieval with &i:,and that speedup could still work with boxed strings instead of symbols.  While I haven't tested the speedup for symbol lookup on fairly small kvs/columns, but don't notice an expense in creating symbols, using boxed strings is unlikely on small kvs to make them not worthwhile.  The key optimization in kv is append based updates/deletes where null code is equivalent to a deleted key.  Ordering and permitted duplicates is a nice feature.

One implementation of RDBMS type tables I believe you want to optimize for is keys fieldnames, fields, unique indexes, non unique indexes,  as boxed shape 1 values. where indexes are sub dictionaries with row key, position value to use when retrieving from fields (inverted table oriented).  In kv, I would just have used symbols for anything that wants an index as automagical simplicity.  Perhaps in your concurrent access focus, you would want row oriented tables?  with only one key/index?  I worry that other applications than your optimized focus will be degraded by removing symbols.  I'm guessing at workarounds needed.

>  I have just now fixed the implementation so that it does not expose any noun that would allow a user to crash the system. You must not use the 16!:n functions that have negative n.

That could be a regression.  get =: dict 16!:_2 is easily transformable to get =: 4: : 'y 16!:_2 x'"1 0   .  It can be good news if it is impossible to crash J by any access to dict.  It;s  bad news if you figured out a way somehow to hide dict inside of object.  It's easy to avoid 1 {:: dict in wrapper functions.  It is harder to never forget (1) 16!:_5 before a temporary dict goes out of scope, and to not have documented causes of crashes.  The latter is the only complexity.  Whether or not you change the internals of get/put, they will probably keep their 16!:_ signatures, and if they don't I can update wrappers.  Whether or not documented, they are exemplified in dictionary class/file.  < dict not crashing (it doesn't) is good for embedding a dictionary as a keyed value.  My only key requests is that < dict continues to work, and (1) 16!:_5 before destruction is not needed, while assuming that a temporary dict created in a function gets erased when function ends.

> It sounds like you want some of your dictionaries to act like regular J values

It is definitely ideal for dictionaries to be first class J values, with 0&{:: to access keys and 1&{:: to access values or just keys/values functions.  Support for 3!:2/3!:1 is 

> such that modifying the dictionary creates a new dictionary with the changes. 

I very much strongly prefer that put returns dict. (in kv, my version of put, set or add, uses a kv as the x argument) I prefer an internal error over some boolean coresponding to key if put fails, because I/anyone else would not know what to do if any individual key failed to update.  It is probably a key/value in wrong format for dict.  If internal threading/locking error, it's still too hard to pick out the key that failed and resubmit just that one, or even if trying all of them again, when to stop retrying.  J users don't want to see C return values, IMO, and it would be ok if all of the put/update command failed if one key failed, assuming again that by far the most likely reason is bad formatting of the one key.

> we want to provide hundreds of millions of batched gets per second
across all threads even while the dictionary is being updated at maximum
speed.  Creating a table of all the keys would be an expensive and
exotic request in such an environment.

accessing all keys or values in such a large dictionary is understandably expensive.  An ideal debugging environment would be for console/REPL to display keys/values in 2 boxes in ,. order (_ 1 shape) and respond to 0 1 {:: as if the dictionary was 2 boxes.  Whether that access needs to be 8 9 {::  (or keys ,&< values) instead, for a technical reason, is not significant to me.  Users can understand that in a production system, requesting the full large table/dict will be expensive/blocking, but there are so many small dict applications, and debugging visualization, where seeing the damn data is going to solve everything quickly.  Ideal would be, even if a pointer/summary to internal data structure, is what is returned as an abstract handle, that it displays keys ,&< values in console, and/or responds to {::

> Functions that I imagine you want are:

> keys - return a table of all keys
> values - return a table of all values
> clone - create a new dictionary that has the same kvs as an old one

clone as a new object I need to destroy is not my preference.  Querying a dict  to filter it as a new dict,whether or not displayable, and whether or not the direct access methods exemplified in jdictionary class, crash J when used outside of them, I'll try to complain about the crashes later.  As long as current 16!: functions are not interacting with jdictionary class/objects in such a way to assume it is immutable, I would understand that a functional approach would work the same despite your warnings. I would only access dict with get,put,del equivalents and not try to bit twiddle through binary representation, and so expect that should work.  It is only 16!:_5 requirement that is troubling.  Single file save/load (3!:1/2) of a dictionary hierarchy is also very convenient (more important than native displayable dictionaries), and one of the major advantages of avoiding OOP approach.  A functional approach makes it easy to apply functions to multiple dictionaries including with t.'', whereas this is difficult (t.'' part) inside a loop.  deep del/get/put syntax can be handled tacitly vs recursive call to explicit function with assignment.  No memory management.  It's important to me that OOP can be avoided, but I'm happy to implement that myself

[Henry Rich]

It seems that you want to work with the full array of keys and values. If that's true, you have no need for the dictionary addon and you should stick with kv. 

I am a little surprised that you mention analyzing chess positions. That very application was once of the use cases i had in mind when thinking about dictionaries. If you use standard J mechanisms, every time you add a key you have to hash the entire key array. When you have a few hundred positions you are spending 99% of your time rehashing old keys. 

In any case, you /must not/use the undocumented 16!:_n functions. They are undocumented for a reason, and they may be changed or removed at any time. They are not a supported API. if you need something, ask for us to add it to the API. 

I think you will find it easy to create a dictionary to replace symbols.

Henry Rich

On Sun, Feb 22, 2026, 10:16 AM 'Pascal Jasmin' via forum <fo...@jsoftware.com> wrote:

> What I think you are asking for is a more gentlemanly dictionary, where
updates are applied one at a time and you can look at the keys or values
at your leisure.  Maybe I can think of it as an offline version of a
general dictionary, one in which updates from other threads are not
possible.

I'll start here, but rest is more deferential.  No.  while small dictionaries are useful, I believe in bulk operations, and espeically helper modifiers to abstract complex tasks.  I believe functional approach to dictionaries makes these functions easier.  I'll repost https://github.com/Pascal-J/kv as my dictionary implementation.  It is meant for high performance, in its own way, but also convenient syntax.  My current project is to create a generic search/solver function generalized from the complexity of a chess search engine.  Dictionaries are a useful means of organizing all of the board state data, and very large data structures can occur in expanding and evaluating all legal moves as nested dictionaries of same fundamental format.  threading is possible as long as each thread explores a single move, and parent thread takes care of updating its key/value for that move/sub dictionary.  A generalized search/solver function is easier with dictionaries than a flat table where legal moves can be 0 1 or many, and expanding the table must guard against each of these edge cases.  Recursive descent on a dictionary structure is an easy path to generalize to all problems... I hope.


 
> the symbols implementation was clumsy and restricted and that we would remove it when we had a good replacement. 

If I had a dictionary with 100 or 1000+ keys, I'd definitely look at using your implementation.I don't actually know if the overall overhead  of using a dictionary with 10-40 keys with symbols (symbol creation overhead vs get savings) is better performance overall than boxed strings.  Instead of high key volume, my kv implementation, has functions to enhance inverted tables, where a key is a field name.  It seems very approachable to build a RDBMS system this way with tables as higher dictionary and join information part of individual table dictionaries.  But then symbols acting as indexed/hash search in one column to get "record" seems useful, and one key per column makes querying easy to read.

while kv is meant to have any keys/values, practically, general dictionaries have symbol keys, and boxed (mixed) values in kv, through autopromotion.  The kvi function is meant to "freeze" a dictionary for retrieval with &i:,and that speedup could still work with boxed strings instead of symbols.  While I haven't tested the speedup for symbol lookup on fairly small kvs/columns, but don't notice an expense in creating symbols, using boxed strings is unlikely on small kvs to make them not worthwhile.  The key optimization in kv is append based updates/deletes where null code is equivalent to a deleted key.  Ordering and permitted duplicates is a nice feature.

One implementation of RDBMS type tables I believe you want to optimize for is keys fieldnames, fields, unique indexes, non unique indexes,  as boxed shape 1 values. where indexes are sub dictionaries with row key, position value to use when retrieving from fields (inverted table oriented).  In kv, I would just have used symbols for anything that wants an index as automagical simplicity.  Perhaps in your concurrent access focus, you would want row oriented tables?  with only one key/index?  I worry that other applications than your optimized focus will be degraded by removing symbols.  I'm guessing at workarounds needed.

>  I have just now fixed the implementation so that it does not expose any noun that would allow a user to crash the system. You must not use the 16!:n functions that have negative n.

That could be a regression.  get =: dict 16!:_2 is easily transformable to get =: 4: : 'y 16!:_2 x'"1 0   .  It can be good news if it is impossible to crash J by any access to dict.  It;s  bad news if you figured out a way somehow to hide dict inside of object.  It's easy to avoid 1 {:: dict in wrapper functions.  It is harder to never forget (1) 16!:_5 before a temporary dict goes out of scope, and to not have documented causes of crashes.  The latter is the only complexity.  Whether or not you change the internals of get/put, they will probably keep their 16!:_ signatures, and if they don't I can update wrappers.  Whether or not documented, they are exemplified in dictionary class/file.  < dict not crashing (it doesn't) is  for embedding a dictionary as a keyed value.  My only key requests is that < dict continues to work, and (1) 16!:_5 before destruction is not needed, while assuming that a temporary dict created in a function gets erased when function ends.

[Jan-Pieter Jacobs]

I've been following this thread with a lot of interest, and have been willing to chime in for a while.

I had a similar initial reaction and reservations as Pascal to removing symbols.

While I'm much in favour of dictionaries and I see that symbols are a sort of subset of dictionaries when used in certain ways, I think the main difference is that with dictionaries (as defined up to now), you can't get a light-weight reference to a key to use elsewhere, the only thing you can get out is the full data that is used as key. Keys inside dictionaries do behave as symbols, but they are only accessible to the dictionary itself, and not to arbitrary user-defined functions, and cannot be used between different dictionaries/arrays as is possible with symbols (as far as I can see).

Symbols on the other hand, are standalone, i.e. they have a global meaning in the session, and make it easy to treat any non-atomic piece of  data (e.g. using 3!:1 if not literal) behave like an atom.

Symbols are currently bidirectional, you can get the symbol from a string and vice-versa.

For a dictionary as it stands now to be used as a set of symbols, it would therefore seem one would need to be able to do reverse-lookup as well: you could store data as keys, with as value an integer serving as reference to the key (or equivalently, the other way around). However, dictionaries now only allow looking up values by keys, not looking up keys by values.

How did you intend current dictionaries to be used as symbols (especially in combination with arbitrary J phrases)? Probably I'm missing something, or I've been using symbols for non-intended purposes all along.

I'm curious to see how dictionaries as integrated J data-type would work; surely some primitives would make sense (like e.). Would this also mean having some sort of symbol (like a key-reference) as type? I guess "key e. dict" (i.e. has__dict) would be clear, as well as dict i. value (reverse lookup, returing the key).

Few days ago, I compared dictionaries with boxed data and symbols for caching a recursive function, and indeed the dictionary approach took only 60% of the time of the symbol-based one.

I want to take a look at my other uses of symbols that I can find back, and will report some comparisons with attempted dictionary-based approaches.

Thanks a lot for the effort of the implementation.

Best regards,

Jan-Pieter

[Henry Rich]

Let's start the code to emulate s: using dictionaries:

require 'dictionary'
NB. create empty symbols table
symb_init_z_ =: {{
symb_z_ =: 0 ". > ('hash' ;< _2 ]\ 'keytype' ; 'boxed') create 'jdictionary'  NB. global symbols dict
strings__symb =: 0$a:  NB. init to no strings
}}
NB. Add the boxed symbol y to the symbols table, return the symbol number
symb_put_z_ =: {{
if. _1 ~: rc =. _1 get__symb y do. rc return.  NB. If already present, return its symbol#
(#strings__symb) put__symb y  NB. The new string gets the next sequential number
#strings__symb =: strings__symb , y  NB. remember the new string
}}
NB. Fetch the symbol# for string y
symb_getsymbol_z_ =: get__symb
NB. Fetch the boxed string for symbol# y
symb_getstring_z_ =: {   ".@'strings__symb'

I am not in a position to test this, but it's a start.  It wouldn't support multithreading.

The suggestions that a dictionary should look like a J value have the wrong end of the stick IMO.  J values are immutable.  When you change one, you have to get a whole new copy.  If you pass a J value into a subroutine, and that subroutine modifies the value as part of its computation, JE makes a new copy.

There are many applications that have a large mapping of keys to values where only a few k/vs are changed at a time, and those changes are passed on as part of the value to subsequent computation.  Marcin has given some good examples, and the chess-position hash is another.

The dictionary addon has get/put/del in O(1) time.  Standard J methods take O(n) time.  n can easily be 100,000 or 100,000,000.  The dictionary addon is 'only' a performance enhancement, but that factor of 100,000,000 is enough to make some tasks feasible that wouldn't be feasible any other way.  Multithreading support is another factor of improvement, if you want it.

The dictionary /must/ bypass the normal J mechanisms for copy-on-change.  It must never copy.  You can make the locale of a dictionary part of a value if you like, but you cannot touch or refer to the dictionary data itself.

Henry Rich

[Pascal Jasmin]

To underscore Jan-Pieter's most important point, there must be some way of using reverse-lookup "getkeys values".  I believe you've already committed to providing keys,values (list all) functions  for dictionary.  If there was special code for e., -:"rank in combination with {::~ #~  with keys/values as arguments, that would be "nice to have".  But getting keys and values is necessary.  Optimization of value lookup is not. 

I saw your "symbols replacement" dictionary.  It is very good (you meant value type as integer I think).  Special nature of that dictionary is a unique to unique key/value relationship.  Implementation can be 2 dictionaries with reverse unique value to boxed string dictionary, and then a "symbol keyed" dictionary just stores integers as key.

A nice (frivolous) feature of symbols as keys is that they look so much better in console display (take up less room) than boxed strings.  This can enhance my ky module by using a private symbol list to each dictionary or dictionary hierarchy.(recursive dictionaries (as functional objects) within dictionaries but all using same "private" symbol list).  I can reimplement kv, with 2 additional columns, where object symbols are fine, and sym->int and reverse int-> sym being the 2 extra columns.  This allows even nested dictionaries to access symbol/translation tables "local" to hierarchy.

an ambitious feature request would be instead of eliminating symbols, make incompatible change to s: such that it is an dyadic adverb, where a: or '' as m is the current monad s: but inserts the new symbols into the dictionary x that is a private symbol table (as per your/my enhanced implemenation) and current dyad s: is m =. current/subset of x parameters.  s: doesn't need left rank support, even though it would lose inverse support by being an adverb.  though (k s:) has definable inverse of (_k s:) 

> you /must not/use the undocumented 16!:_n functions. 

with caveat that I understand that future versions of J may change jdictionary class, and that I would have to reimplement base functions, I cannot understand how duplicating jdictionary's create, get, put, has impact on the "private" dict variable inside the class/objects being duplicated as an indepedent variable, can run into problems if I limit all interactions with equivalent get/put/has... functions.  The only problematic area I understand as possible is a user burdened "destroy hoop jump" (16!:_5) instead of internal cleanup guards.  Hopefully, your previous announcement means 16!:_5 necessity no longer exists.

>  am a little surprised that you mention analyzing chess positions. That very application was once of the use cases i had in mind when thinking about dictionaries. If you use standard J mechanisms, every time you add a key you have to hash the entire key array. When you have a few hundred positions you are spending 99% of your time rehashing old keys. 

My implementation may be very different than what you (maybe flat dictionary that uses move list as keys?) envisioned.  Mine is that every legal move for a board state is a key to a new sub kv that holds the new board state with its own legalmoves list, and its own potential sub kvs.  This means a very small (< 100 always) number of keys, and because memory optimization is critical, I could consider padded strings as keys, use the output of ($ keys) to pad my keylookup function.  The smaller the universe of keys, the less important a symbol optimization becomes.

While I have no idea about "time rehashing keys" analysis, a design error I made with kv is to not limit keys to either symbol or numeric.  Current implementation will allow mixed symbol and numeric keys, that then require boxed symbol/numeric PITA access, that isn't important because user will just convert their mixed numeric keys into symbol of stringified numeric keys for sanity instead.  Symbols or numeric as only key options does not seem like a limiting option, and your implementation of explicit key/value types is good, where if you wanted to allow floating point keys/values, then declare that at begining, and prevent errors if you wanted just integers.  My point in this paragraph is that symbols as a key is (or with implementation similar to your example) a very useful universal key type, despite potential overkill in some situations.

[Henry Rich]

If you need reverse lookup, have a second mapping from value back to
key.  Simple as that.

destroy is & will always be necessary.

s: is slated for removal, but the dictionary replacement is serviceable
as you see.

If you want to work with the full list of keys/values, the dictionary
addon is not for you.  Stick with kv.

We will have load and unload functions that you can use to fetch the
current set of keys/values.

Henry Rich

[Pascal Jasmin]

There is also an essential need for reverse lookup when values are non-unique.  values can be keys to something else.  "For keys were 1 = f values do something with keys/values".  You/we could consider dictionary as an implementation for row oriented RDBMS type table.  

> We will have load and unload functions that you can use to fetch the current set of keys/values.

This could mean a binary format similar to 3!:1/2?  Only with 'tree' type it is possible to get all keys/values right now (I posted earlier in thread getall function).  Being able to physically see that any "sophisticated" (or naive) functions (or if there is anything you forgot to do) do what you intended is very important to most workflows.

> destroy is & will always be necessary.

my only request is that destroy function not require 16!:_5 call.  If it is always necessary, then it could always be done as an internal check.


will there still be a "global" dict variable in jdictionary?

[Henry Rich]

Keys must be unique. If you want to repeat a key, you need to extend it with a disambiguating field. 

destroy is necessary. Whether it executes 16!:_5 or any other function is up to the implementation and subject to change. 

Likewise with the dict variable. It is not part of the API and is subject to change. It would be foolhardy to look at it, because it won't show you keys or values and what it does show may change.

hhr

[bill lam]

Does hash in dictionary handle collision?

[Henry Rich]

Yes. 

Henry Rich

[Raul Miller]

Another possible implementation of "non-unique keys" would be to add a
layer of structure such that the "non-unique key" references a
corresponding list of values.

The details, of course, would depend on why this is needed.

--
Raul

[Pascal Jasmin]

the need to lookup by value is as obvious as sql having the ability to query on non-primary key.fields.  It doesn't need to be fast or sped up, or even thread safe, or to be milisecond accurate if a different thread might be updating data.  A snapshot of values must be available.

To summarize the unique->unique mapping of symbols replacement.  2 dictionaries are used that point to each other's data in an inverted way.  A boxed string is matched with an integer, and we can use the integer code instead of a symbol in other dictionaries/applications, but need the reverse lookup to display/retrieve the meaning of the integer code.

Consider a RDBMS employee table.  We use an Employee_id (integer) as the key because multiple people could be called John Smith.  A rolodex/contacts app still needs to lookup by people's names.  While such search doesn't need to be optimized, it get's a bit crazy to have a bidirectional symbol dictionaries (2) for each of first middle last names, and then only complete name searches get optimized.

The 'tree' dictionary format does seem to permit "search while typing" with range query, essentially making the first field in a list of fields as values, a search optimized field.  This is at expense of employee_id lookup 'hash' structure that optimizes for "link this exact employee to some other table/information".  

There are improvements to symbol system from dictionary techniques, mainly through tree type to provide partial search indexing, but it costs indirect storage, and dereferencing  bidirectionally to actual values.  Even if you get quick access to the integer code representing the value you are searching, you need to find the indexes of the code where they are stored, or store the index (instead of integer code) in the dictionary with complications for delete/edits.

While symbol replacement strategy is very flexible, it requires a lot of work, and doesn't negate need to query by value in simple unoptimized way.

[Raul Miller]

Lookup by value could be done like Henry indicated, with a pair of
dictionaries (one for the initial map, the other for the reverse map).
If that's not viable I (or probably anyone) would have to understand
what problem was being addressed.

You could, of course, do a space time tradeoff and walk through every
key in one dictionary, comparing values to the target value. But,
again, if that's the desired solution why is that desired?

--
Raul

On Wed, Feb 25, 2026 at 5:25 PM 'Pascal Jasmin' via forum

[Pascal Jasmin]

There are complex extra steps when values are not guaranteed unique, which is the more common dictionary format that my lengthy example analogized.

[Raul Miller]

This sounds like an argument for building an addon to address these
issues (possibly several, since complexity hints at ambiguity and
different implementations might prefer different resolutions for that
ambiguity).

--
Raul

On Thu, Feb 26, 2026 at 12:01 PM 'Pascal Jasmin' via forum

[Henry Rich]

Yes. The basic making function may not be enough for some applications. 

Henry Rich

[Marcin Żołek]

Relational databases and dictionaries serve different purposes. Could you share more details about your use case so we can help you decide whether dictionaries, Jd, or ODBC would be the best option?

Marcin

[Marcin Żołek]

[Pascal Jasmin]

I pack too many points into comments, that distract from main point.  I do not have a pressing personal need for a table implementation.  In passing, I noted that a table can be a simple extension of a dictionary.  In Jd, a dictionary is just a 2 column table with first column uniquely indexed. Both Jd and apparent direction for dictionary take an anti-J view of "side effects" modifying in place by default, although Jd has the common sense design of letting users see/query by data, and a means to copy a table.

Like , and }'s copy by default, but optimize in place if name assigned to same y, I feel that would make the best design, and one  consistent with rest of J language.  I can settle for an explicit copy/clone function, and continued ability to box the internal representation stored in dict variable.

On Friday, February 27, 2026 at 06:10:06 a.m. EST, Marcin Żołek <marcin...@students.mimuw.edu.pl> wrote:






Relational databases and dictionaries serve different purposes. Could you share more details about your use case so we can help you decide whether dictionaries, Jd, or ODBC would be the best option?


Marcin

[Henry Rich]

The dictionary does what it does precisely because those are important functions that are slow in standard J. The restrictions on access to internal information stem from implementation decisions that make the dictionary as fast as possible in heavy use. 

In the heaviest use case, with multiple threads simultaneously modifying the dictionary, added features such as reading all the keys and values would be prohibitively slow.

This thread has shown that there might be a call for such added features when a dictionary is single-threaded. They would be easier to implement single-threaded. We will keep these in mind after 9.7 is released, as we all get a better idea of what features are required. 

Henry Rich