Regular Expressions not thread safe?

[Thomas McGuire]

I was playing with byte pair encoding (BPE) on a decent sized file of 500,000 paragraph sized stories. I read the stories in as a boxed list of strings called ’storylist'. The encoder script from the picoGPT-in-j relies on regular expressions for the pretokenization step.

I have expanded the code to use mutexs as follows: 

encode =: {{

pat1 =. '(*UTF)(*UCP)''s|''t|''re|''ve|''m|''ll|''d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+'

11 T. mutex NB. acquire mutex

pretok=. pat1 rxall utf8 y

13 T. mutex NB. release mutex

; {{vocab_i bpe cs {~ (bs{a.)&i. y}} each pretok

}}

Now this works but it is technically bottlenecking when doing the 'pretok=. pat1 real utf8 y’.

I placed pat1 within the encode since it was originally global and I thought I was stepping on memory I had some control over. But the pattern

inside the function or outside I get the same error.

without the mutex statements I get the following error when I run the encoder in parallel using parallel each (‘peach’)

peach=: (t.'')(&>)

encode__encoder peach storylist

|index error in rxfrom, executing dyad <;.0

|starting index out of bounds (value=97303, axis len=648) in cell of x with path 7

| ;{{vocab_i bpe cs {~ (bs{a.)&i. y}}each pat1 rxall utf8 y

Press ENTER to inspect

Wondering if anyone has some intimate knowledge of the RE implementation and knows if this may be fixable or is there just too much common memory that will get stepped on in a multithreaded environment.

rxall is shorthand for an rxmatches rxfrom combination (hence the error mentioning rxfrom), the ‘cut' dyad (<;.0) is part of the rxfrom definition so not sure why that would be a problem and it’s doing a reverse adverb.

Tom McGuire

[Henry Rich]

I don't have intimate knowledge of jregex but it is easy to see that regcomp_jregex_ (called by rxmatch which is called by rxall) is not threadsafe.  It sets the global names lastpattern, lastcomp, etc. which are referred to later in that verb and in other verbs.

Since all the threads share a namespace, the values of these names are shared, with unhappy results.

The addon needs rewriting for multiple threads, either

* to eliminate the public assignments
* to allow execution in a user locale to keep the public assignments in different threads separate.

Henry Rich

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Thomas McGuire]

To your second starred item. Do you mean making it more object oriented so that I would assign a different RE instance to each thread thereby giving them their own memory space?

Tom McGuire

[Henry Rich]

Yes.  Since your application doesn't need to share globals between tasks, this would suffice for your problem.

You could create locales with

cocreate 'rxtask'

where

cocreate_rxtask_ =: {{
coinsert 'jregex'
}}

It seems to me that if you execute rxall when jregex_ is in the search path, it will call rxall without setting the locale to jregex.

You could then execute your searches in the locales.

It might make sense to compile the pattern first, but would that be in each locale or in jregex?

Cheap talk.  As the saying goes: Nothing is impossible, as long as somebody else has to do the work.

Henry Rich

[Henry Rich]

On reflection, if you have a great many tasks it will be a problem to ensure that they keep separated.  Perhaps you could have each task create a locale to run in.

Henry Rich

On 6/15/2025 8:06 AM, Thomas McGuire wrote:

[Thomas McGuire]

Here is a twist to this problem. One person of our forum private messaged me about trying this on j9.6. I have my old j9.6-beta22 intact from when I switched over to j9.7 and was able to distill the issue down to a handful of J lines. This seems to work on j9.6 where it crashes on j9.7. I will explain one caveat I didn’t test the data integrety of the j9.6 answer just that it didn’t error out. I suppose it is possible I am overwriting valid areas of memory with just the wrong data as opposed to the error I get in j9.7. 

Here is the trimmed down code that elicits the error on JQt j9.7-beta3:

setwd=:1!:44

pwd=:1!:43

setwd jpath '~/devLLM/picoGPT-in-j'

load 'encoder.ijs' NB. from picoGPT-in-j directory

load 'utils.ijs' NB. from picoGPT-in-j directory

MODELS_DIR

models

encoder =: MODELS_DIR conew 'encoder'

Loading tokenizer...

Reading merges.txt

Reading vocab.json

Processing vocab

Building lookup verbs

Done.

{{0 T.0}}^:] <: {. 8 T. ''

11

peach=: (t.'')(&>)

storylist =: 10000$<556$'Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country.'

encode__encoder peach storylist

|index error in rxfrom, executing dyad <;.0

|starting index out of bounds (value=_6710525949, axis len=556) in cell of x with path 1

| ;{{vocab_i bpe cs {~ bs i. a. i. >y}}each pat rxall utf8 y

Press ENTER to inspect

If you were inclined to duplicate this you would need to download the picoGPT-in-j repository from GitHub (https://github.com/NPN/picoGPT-in-j). 

This code works on j9.6 and the timings show a speed up using ‘peach' over ‘each’.

I went over release notes and I don’t see anything about changes to regex. The website info on regex indicates we went over to PCRE2 with j807.

Tom McGuire

[Thomas McGuire]

I take it back I think the timespacex was masking an error that just happened to not crash J under JQt9.6. After trying it in Jconsule 9.6 it crashed and now I can’t reproduce anything working in J9.6 even under JQt. 

Which makes sense I went through the GitHub logs and no changes have been made to regex for 2 years. 

So sorry for the waste of bandwidth. Back to the answer being regex is not thread safe. 

Tom McGuire

[Ak]

so the time spacex was returning a result which was the error message?

[bill lam]

pcre2 itself is threadsafe. Did you try encapsulate jregex into a
numbered locale in each thread?

[Thomas McGuire]

> On Jun 16, 2025, at 2:34 AM, bill lam <bbil...@gmail.com> wrote:
>
> pcre2 itself is threadsafe. Did you try encapsulate jregex into a
> numbered locale in each thread?
>

That’s my next step. looking at my code I use only one encoder object and it’s making a direct call into rxall.

It will take some designing, though, as I have 500,000 stories in any given list and I will likely need a pool of objects to operate on them (I don't want to create/destroy 500k different instances of an object). There will need to be some mutexing since I want each thread to complete a given task before context switching to the next task.

So instead of my nice trivial

<verb> peach <list of 500k strings>

I will probably have to divy up the list of 500k strings and run a new thread safe verb against each subsection.

This also brings up the question is there a thread id that you can query? It would be nice to link a thread id to a particular object instance. But I don’t see that I can tell which thread I am in once the task starts running.

Tom McGuire

[Ed Gottsman]

Tom,

I believe 3 T. ‘’ will give you the id of the currently-executing thread.

Ed

[Henry Rich]

The regex addon will produce incorrect results, as I mentioned before.  It shouldn't crash, though.  Please give me the best script you have that crashes.  I cannot work on it until Saturday.

Henry Rich

[Thomas McGuire]

On a macbook pro M2 max chip using Jconsole

   9!:14''

j9.7.0-beta3/j64arm/darwin/commercial/www.jsoftware.com/2025-04-03T02:17:42/clang-15-0-0/SLEEF=1

NB. I have distilled this down to a few lines of J code

   require 'regex'

      pat =: '(*UTF)(*UCP)''s|''t|''re|''ve|''m|''ll|''d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+'

      storylist =: 10000$<556$'Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country.'

   {{0 T.0}}^:] <: {. 8 T. ''                                                   11

   peach=: (t.'')(&>)

   pat&rxall peach storylist

JE has crashed, likely due to an internal bug.  Please report the code which caused the crash, as well as the following printout, to the J forum.

|index error in rxfrom, executing dyad <;.0

|starting index out of bounds (value=_136784836, axis len=556) in cell of x with path 15

|       pat&rxall peach storylist

   Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

-----------------------------------------------------------------------------

Abort trap: 6

logout

Saving session...

...copying shared history...

...saving history...truncating history files...

...completed.

[Henry Rich]

Thanks. I'll look on Saturday. It's an unusual crash: it detects the index error, analyzes it, prints the message, and then crashes on the way back to the console prompt.

Henry Rich 

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Henry Rich]

I'm pretty sure this is a simple JE crash caused by the index error. The code has a blunder in the error path that would corrupt the system. 

The bug showed up because of the sharing of global names in jregex. One thread set a value that was picked up by another thread. 

I will fix the crash when I get home, for the next beta. 

Henry Rich 

[Henry Rich]

I was wrong thinking that there was a blunder in <;.0 .   I think the error comes from the regex addon.

The addon uses globally-named boxed data areas to hold workareas created by the pcre2 package.  The global naming means that the same workareas may be in use in multiple threads simultaneously.  That could lead to anything.

We need to rewrite the regex addon so as not to use global names.

Henry Rich

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Thomas McGuire]

For my purposes I have scaled the functionality back. I am not relying on multiple compiled patterns that the regex library has made accommodation for with global variables such as “last comp”, “lastpattern”,”lastmatch”. However there is still the issue that “lastmatch" holds a pointer from where pcre2  places the match results. I am still working on a solution but by using the object oriented features of J. I’m thinking that by using a mutex in a rewritten “rxmatches” equivalent I can set up an object per thread. This way I complete processing a string before moving on to the next string. 

The following code works in a single threaded mode. I need to put in the mutex statements and then figure out how to spread them out to a particular thread. There is also some superfluous stuff in my “ptmatches” that is probably no longer needed especially the x argument. p and n are just the pattern string which is already compiled. 

NB. pretok.ijs - wraps regex in a semi thread safe manner

NB. use in place of regex for Byte Pair Encoding pretokenization

NB. it will use a regular expression and uses some of regex.ijs

NB. under the covers

require 'regex'

coclass 'jpretok'

create =: 3 : 0

NB. y holds the arguments used to creat the class

NB. assignments made inside verbs are private to the instance

ptpattern=: y

msg=. ,2

off=. ,2

flg=. (PCRE2_UTF_jregex_*RX_OPTIONS_UTF8_jregex_)+PCRE2_MULTILINE_jregex_*RX_OPTIONS_MULTILINE_jregex_

ptcomp=: 0 pick rc=. jpcre2_compile_jregex_ (,y);(#y);flg;msg;off;<<0

'msg off'=. 4 5{rc

if. 0=ptcomp do.

regerror msg,off

return.

end.

pthandle=: 0

ptmatch=: 0 pick jpcre2_match_data_create_from_pattern_jregex_ (<ptcomp);<<0

ptnsub=: 0 pick jpcre2_get_ovector_count_jregex_ <<ptmatch

EMPTY

)

destroy =: 3 : 0

NB. release any resources acquired

codestroy'' NB. release the instance

)

NB. =========================================================

ptmatch1=: 3 : 0

ptmatchtab 0 pick jpcre2_match_jregex_ (<ptcomp);(,y);(#y);0;0;(<ptmatch);<<0

)

NB. =========================================================

ptmatch2=: 3 : 0

's p'=. y

ptmatchtab 0 pick jpcre2_match_jregex_ (<ptcomp);(,s);(#s);p;PCRE2_NOTBOL_jregex_;(<ptmatch);<<0

)

NB. =========================================================

NB. get match table

ptmatchtab=: 3 : 0

if. y >: 0 do.

p=. 0 pick jpcre2_get_ovector_pointer_jregex_ <<ptmatch

'b e'=. |:_2 [\ memr p,0,(2*ptnsub),4

_1 0 (I.b=_1) } b,.e-b

elseif. y=_1 do.

,:_1 0

elseif. do.

regerror y

end.

)

NB. =========================================================

ptmatches=: 4 : 0

echo x

'p n'=. 2 {. boxopen x

echo p

echo n

NB. regcomp p

NB. ACQUIRE MUTEX HERE

m=. ptmatch1 y

if. _1 = {.{.m do. i.0 1 2 return. end.

s=. 1 >. +/{.m

r=. ,: m

while. s <#y do.

if. _1 = {.{.m=. ptmatch2 y;s do. break. end.

s=. (s+1) >. +/ {.m

r=. r, m

end.

if. #n do. n{"2 r end.

NB. RELEASE MUTEX

)

ptfrom=: ,."1@[ <;.0 ]

ptall =: {."2@ptmatches ptfrom ]

mypretok =: 0&".@> pat conew 'jpretok'

pat& ptall__mypretok 'Now is the time for all good men to come to the aid of their country'

(*UTF)(*UCP)'s|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+

(*UTF)(*UCP)'s|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+

┌───┬───┬────┬─────┬────┬────┬─────┬────┬───┬─────┬───┬────┬────┬───┬──────┬────────┐

│Now│ is│ the│ time│ for│ all│ good│ men│ to│ come│ to│ the│ aid│ of│ their│ country│

└───┴───┴────┴─────┴────┴────┴─────┴────┴───┴─────┴───┴────┴────┴───┴──────┴────────┘

[Henry Rich]

I am thinking there is an easier way.

   create_jregex_ =: ]
   destroy_jregex_ =: 18!:55@coname
   rx =: 0 ". > conew 'jregex'
   rx
1
   'abc' rxmatches__rx 'abcabcabc'
0 3

3 3

6 3
   lastcomp__rx
2145731581808

This lets you run all the regex verbs, but in the specified locale.  If you start each task with

rx =: 0 ". > conew 'jregex'

and then call all regex verbs in the task with __rx as a suffix, you should get namespace separation between tasks.

Untested.

Henry Rich

[Henry Rich]

Bill Lam has pointed out to me that create and destroy are not needed for this application.

Henry Rich

[Thomas McGuire]

If I am using the same pattern regex has code to reuse the compilation and the result area so I’m not sure this will work without the create and destroy. 

Tom McGuire

[bill lam]

The application itself needs not be OOP. The basic requirement is regex has its own locale in each thread.

[bill lam]

If you serualize using mutex, it will defeat the purpose of running in parallel.

On Mon, Jun 23, 2025 at 7:17 AM Thomas McGuire <tmcgu...@gmail.com> wrote:

[Henry Rich]

regex remembers patterns in J globals.  By putting each task in its own locale (with conew) you keep those globals separated.

conew creates the numbered locale.  create and destroy are never called in this simple application.  codestroy or coerase will delete the locale at the end.

Henry Rich

[Thomas McGuire]

If I use objects and create locale variables to hold the regex info I can serialize on a per thread basis, each thread will process one of the strings from my “peach”. Since it’s a single pattern I run on all the strings I will process I can dedicate a separate object to each of the threads. I should be able to get strings to spread out across threaded objects and get some parallel benefit. The downside is it will take some coding to make this happen. But I agree if it is not designed correctly I could end up with all the strings serialized on a single thread and then I’m just doing the same thing as no threading.

Tom McGuire

[Raul Miller]

I don't think the overhead of locales/objects is desirable here.

If I understand correctly, you need one instance of the compiled
regexp per thread. Presumably the compiled instance holds state
information during the evaluation of a regex match.

That said, it looks like rxcomp explicitly prevents this from
happening. So I am not sure I understand the difference between the
tests that succeeded and the tests that failed.

--
Raul

[Henry Rich]

The problem is not just the compiled regexp, which could probably be
shared.  The regex package uses public names for internal
communication.  When you call rxmatches, it calls regcomp which creates
4 public names, to hold the pattern, the matches, and info about the
matches.  These public names are then filled by calls to pcre2 and
further used by rxfrom to copy the data.  This is all workable in a
single-threaded system.

With multiple tasks, the tasks are writing to the same public names
simultaneously.  This is chaos: one task stores an array into a name and
then passes the array into pcre2 by address. While pcre2 is running,
another task assigns the name, which frees the block pcre2 is writing. 
That block gets reused while pcre2 is writing to it.  Crash.

Numbered locales don't cost much.  All we need here is a way for each
task to keep its public names to itself.

Henry Rich

[Thomas McGuire]

On Jun 24, 2025, at 9:42 AM, Henry Rich <henry...@gmail.com> wrote:

The problem is not just the compiled regexp, which could probably be shared.  The regex package uses public names for internal communication.  When you call rxmatches, it calls regcomp which creates 4 public names, to hold the pattern, the matches, and info about the matches.  These public names are then filled by calls to pcre2 and further used by rxfrom to copy the data.  This is all workable in a single-threaded system.

With multiple tasks, the tasks are writing to the same public names simultaneously.  This is chaos: one task stores an array into a name and then passes the array into pcre2 by address. While pcre2 is running, another task assigns the name, which frees the block pcre2 is writing.  That block gets reused while pcre2 is writing to it.  Crash.

Yeah that’s what I found out. As I tried diffeerent things the race conditions may not always conflict to cause a crash. But then I need to make sure I’m not just shuffling data into the same memory from different threads (The chaos you speak of). That’s why I was trying to organize this into a per thread basis and mutex to complete a full string processing operation before releasing a thread for the next string. While that is a bunch of coding overhead it’s simple enough for my pea brain to comprehend the multithreading. 

Numbered locales don't cost much.  All we need here is a way for each task to keep its public names to itself.

OK so here is a question. I am going to process 500k moderately sized strings. is there a reason why I could not use a different locale for each string. I would ‘conew’ then ‘coerase’ after being finished with the locale. Then I would just ‘peach’ the verb containing the regex code on the boxed list of strings. So while I haven’t had time to try it yet I’m thinking of the following: 

rgx =. 0&".@> conew 'regex'

pretok=. pat1 rxall__rgx utf8 y

coerase rgx

This would be inside a J verb that is performing the encoding I am looking for.

Henry Rich

On 6/24/2025 9:20 AM, Raul Miller wrote:

I don't think the overhead of locales/objects is desirable here.

If I understand correctly, you need one instance of the compiled
regexp per thread. Presumably the compiled instance holds state
information during the evaluation of a regex match.

That said, it looks like rxcomp explicitly prevents this from
happening. So I am not sure I understand the difference between the
tests that succeeded and the tests that failed.

If Henry’s suggestions are not enough it is easy to rewrite a separate rxmatches that relies on a single compiled pattern and doesn’t try to store multiple patterns. I can do this by reusing some regex code in a new object. The only problem is that I can’t get rid of the ‘results area’ requirement of the PCRE2 routines. So I would need a mutex per thread to isolate the writing to the ‘results area’ of PCRE2. 

In terms of over head it really comes down to a little bit of code. Since my machine only gives you 11 threads, I would only be creating 11 objects all done prior to threading out the main processing. The objects would be kept in an array that I would use the thread number (3 T. ‘’) to select the object. I think with a mutex I could get all the string tasks to spread out across the threads with ‘peach’ (parallel each). 

This is my back up if Henry’s locale trick with regex doesn’t work. 

[Henry Rich]

What you have there is exactly what I had in mind, EXCEPT that I would do batches of strings rather than 1 locale per string.  That is,

rgx =. 0&".@> conew 'regex'

pretok=. pat1&rxall__rgx@utf8&.> y
coerase rgx

where you have put, say, 50k strings into each y.

A numbered locale has a table for its defined names and the contents of those names.  It's not huge, but perhaps bigger than a single string.

If you have one task per string, the system will create all 500k tasks and set the threads loose on them.  Each thread will

* arbitrate for the next task
* create the numbered locale it will use
* compile the pattern & define work areas
* run the regex search
* tear down the locale

That's a lot of overhead per string.  Good design with multithreading is to make the individual tasks are heavy as they can be, where the metric is

 (time required not counting threading overhead)/(# bytes of input/output).

In your case the metric is the same no matter how many threads you have per task, so you should reduce overhead by having many threads per task.

If your threads require varying amounts of processing time, you might create more tasks than threads so that you don't get stuck with one long-running task holding up completion.

Henry Rich