Reference Standard M V1.75.0 has been released!

[David Wicksell]

This release has improvements to stability, documentation, the build process, and many more.

https://gitlab.com/Reference-Standard-M/rsm

David Wicksell
Fourth Watch Software LC

[K.S. Bhaskar]

Congratulations, David!

Regards
– Bhaskar

[OldMster]

On Thursday, November 18, 2021 at 5:46:11 PM UTC-5, David Wicksell wrote:

Congratulations David!

[pahihu]

Hi,

Random writing of globals is about 4-5 times slower than in Ray's MV1 version.
After executing the routine below, integrity checking reports
"dbc is too big - overflows block".

I've created the DB: rsm -v TEST -b 16 -s 50000 -m 128 testdb
Started the environment: rsm -j 256 -g 256 testdb

And executed the routine below: D ^UPSCALE,IDX^UPSCALE

Regards,
pahihu

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Saved by %RS from [MGR,TEST] on 10 Nov 2021 at 14:44:14 PM

UPSCALE
UPSCALE ;UpscaleDB benchmark;AP;[2016-04-10 15:58]
n cnt,i,st,et,tmp,x,val,D,E,N s D=",",N=$G(NN,1000000)
w "Cleaning up...",!
k ^rec
; === WRITE ===
w "Writing ",N," records...",!
s E=N/10
s st=$H f i=1:1:N s ^rec(i)=1+$R(E)
s et=$H,et=$P(et,D,2)-$P(st,D,2)
w "Elapsed time ",et," secs",!
; === READ ===
w "Read ",N," records...",!
s cnt=0
s st=$H f i=1:1:N s cnt=cnt+1,val=^rec(i)
s et=$H,et=$P(et,D,2)-$P(st,D,2)
w "Elapsed time ",et," secs",!
w "Count ",cnt,!
; === MAX ===
w "MAX() ",N," records...",!
s max=^rec(1)
s st=$H f i=2:1:N s val=^rec(i) s:val>max max=val
s et=$H,et=$P(et,D,2)-$P(st,D,2)
w "MAX() = ",max,!
w "Elapsed time ",et," secs",!
w "Count ",cnt,!
Q
IDX ;Indexing;AP;[2017-12-11 15:01]
n cnt,i,st,et,tmp,x,val,D,N s D=",",N=$G(NN,1000000)
w "build index for ",N," records...",!
k ^xrec
s st=$H f i=1:1:N s val=^rec(i),^xrec(val,i)=""
w "MAX() = ",$O(^xrec(""),-1),!
s et=$H,et=$P(et,D,2)-$P(st,D,2)
w "Elapsed time ",et," secs",!
Q

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[David Wicksell]

Thank you pahihu.

When I test performance versus MV1 on my various systems, they are pretty
comparable, even on this new version. I haven't been focusing on optimization
yet, as there were so many bugs and other issues with stability that I was
working on. I'm also focused on implementing the entire 1995 standard, for the
MDC. When running your test code, MV1 is a second or two faster in the
performance testing, but actually several seconds slower than RSM in the
indexing test.

However, you are right about the integrity issues. I do a bunch of testing, and
try to catch regressions, but until I have a fully developed test suite, or the MDC
updates the MVTS (M Validation Test Suite) enough for me to use it, some things
might slip from time to time.

It appears that I introduced that regression between RSM V1.73.0 and V1.74.0.
So if you do a `git checkout v1.73.0` and re-run them, followed by an integrity
check, it will come out clean.

I'm also chasing a bug with do/go offsets that was introduced with V1.75.0,
which I'm close to fixing. I will, of course, make sure to fix this dbc issue as well.
It's possible that the bug is in the integrity checker, and not the database module,
as I have already fixed one or two bugs that flagged integrity issues that were not
there, so it might just be a false positive. Of course, it might be real too, though it's
weird that I haven't seen it with any of the global testing that I've done.

Thanks again for letting me know, and hopefully I'll have a new version out that fixes
these issues soon.

[pahihu]

David Wicksell ezt írta (2021. november 20., szombat, 20:50:19 UTC+1):
> I haven't been focusing on optimization yet, as there were so many bugs
> and other issues with stability that I was working on.

The performance of MV1 is shaped by its design. IMHO stability is more
important than performance.

> When running your test code, MV1 is a second or two faster in the
> performance testing, but actually several seconds slower than RSM in the
> indexing test.

Yes, it could depend on the system configuration: on my system RSM does
the indexing in 146 seconds and MV1 finishes in 36 seconds.

> It's possible that the bug is in the integrity checker, and not the database module,
> as I have already fixed one or two bugs that flagged integrity issues that were not
> there, so it might just be a false positive.

Sure, there are hidden gems in the code base.

Congratulations for your achievements so far! Good luck!

pahihu

[David Wicksell]

Thanks pahihu.

Yep, my gut feeling was correct. It's a bug in the integrity checker, and I introduced it by mistake
a couple versions ago. The eob (end-of-block) check is off by one, flagging dbc (data block counter)
overflow errors that are not there. It is already fixed, and once I fix one more thing, I'll push out a new
release, V1.75.1. So there are no actual integrity issues there after running your code. Thank you.

[Terry Wiechmann]

I need to congratulate David on a job well done. I worked closely with Ray on MV1 for a few years as a tester of MV1 (something to do in my retirement). I came to know it rather intimately. We got it to a relatively stable state. However, MV1 in it's original form was what I would call "minimalist". Without elaborating, let me say I was really happy to hear that David had taken over the project. Since taking over he has made numerous foundational, structural and operational changes that, IMHO, will make it easier for him to build on and the RSM programmer to use (name and string length increases, more buffers, larger databases,etc.) So, large changes are invariably going to result in bugs. My experience with David is he responds immediately to these "challenges".
I encourage everyone to take a look at the RSM implementation and help shake it down. Again, big thanks to David.

[David Wicksell]

Thank you Terry.

[David Wicksell]

Hello everyone,

I have pushed out a patch release (V1.75.1) of RSM to fix the issues that were
reported in V1.75.0. Because of a small bug in the bytecode format in V1.75.0,
you might need to recompile your routines, once you upgrade to V1.75.1. You
can do so easily with `do ^%RECOMP`.

One thing I haven't really mentioned is that I maintain a branch in RSM where I
have implemented a number of the MDC Type A extensions that were finished
and set to be part of the Millennium Draft Standard in 1999. Since that standard
was never ratified, these features never made their way in to many of the M
implementations. So I've been adding the extensions that I think are worth trying
out, for a new M Standard, should we ever get there. These extensions, being Type
A, will be the first inclusions in a new standard.

So if you are interested in playing around with them, you can checkout the
`m-2021-standard` branch in the RSM repo. A few of the features that have been
added and are available in RSM V1.75.1 T1 (the T1 marks this as a test version)
are as follows:

1) MDC type A extension X11/95-31 Kill Indirection
This implements argument indirection for the exclusive kill command:
RSM [Dwarfman]> set a="@b",b="x",x=1
RSM [Dwarfman]> write $&%zwrite
a=@b
b=x
x=1
RSM [Dwarfman]> kill (@a)
RSM [Dwarfman]> write $&%zwrite
x=1
RSM [Dwarfman]>

2) MDC type A extension X11/94-47 New SVN Addition: $Test
This implements new $test:
RSM [Dwarfman]> if 1 xecute "if 0" write $test
0
RSM [Dwarfman]> if 1 xecute "new $test if 0" write $test
1
RSM [Dwarfman]>

3) MDC type A extension X11/94-4 Two Character Operators
This implements new operators (>=, <=, ]=, and ]]=):
RSM [Dwarfman]> write 5>=4
1
RSM [Dwarfman]> write 5>=5
1
RSM [Dwarfman]> write 5>=6
0
RSM [Dwarfman]> write 5<=4
0
RSM [Dwarfman]> write 5<=5
1
RSM [Dwarfman]> write 5<=6
1
RSM [Dwarfman]> write 5]=5
1
RSM [Dwarfman]> write 5]=10
1
RSM [Dwarfman]> write 5]=50
0
RSM [Dwarfman]> write 5]]=5
1
RSM [Dwarfman]> write 5]]=10
0
RSM [Dwarfman]> write 5]]=50
0

4) MDC type A extension X11/97-22 Set $Qsubscript Pseudo Function
This implements the ability to use set $qsubscript, to build up name references:
RSM [Dwarfman]> set name=$name(^name(1,2,3))
RSM [Dwarfman]> write name
^name(1,2,3)
RSM [Dwarfman]> set $qsubscript(name,0)="^global"
RSM [Dwarfman]> write name
^global(1,2,3)
RSM [Dwarfman]> set $qsubscript(name,2)="two"
RSM [Dwarfman]> write name
^global(1,"two",3)

5) MDC type A extension X11/93-39 $Reference
This implements the ability to set $reference:
RSM [Dwarfman]> write $reference
RSM [Dwarfman]> write $data(^global)
0
RSM [Dwarfman]> write $reference
^global
RSM [Dwarfman]> set $reference="^name(1)"
RSM [Dwarfman]> write $reference
^name(1)
RSM [Dwarfman]> kill ^garbage(1,"two",3)
RSM [Dwarfman]> write $reference
^garbage(1,"two",3)
RSM [Dwarfman]> set $reference=""
RSM [Dwarfman]> write $reference
RSM [Dwarfman]>

6) MDC type A extension X11/96-27 Xor Operator
This implements the exclusive or operator:
RSM [Dwarfman]> write 0!!0
0
RSM [Dwarfman]> write 0!!1
1
RSM [Dwarfman]> write 1!!0
1
RSM [Dwarfman]> write 1!!1
0
RSM [Dwarfman]> write 0'!!0
1
RSM [Dwarfman]> write 0'!!1
0
RSM [Dwarfman]> write 1'!!0
0
RSM [Dwarfman]> write 1'!!1
1

There are a few other minor corrections to some functions and error codes and
such, and more Type A extensions will be added over time. This might not matter
to most people, but one of the goals of a reference implementation, which is what
RSM is, is to provide a playground to play around with new ideas in the language, to
see how folks like them. I will also be adding the Open M Interconnect (OMI) and M
Windowing Application Programming Interface (MWAPI) standards, as well as much
more, over time. Thank you.

[Flávio Fornazier]

Hi Mr.David,

Congratulations for the great job!

Could you point me to OMI and MWAPI documentation that you intend to implement on RSM?

Thank you very much! =)

Regards.

[David Wicksell]

On Friday, December 3, 2021 at 9:41:49 PM UTC-7, flaviof...@hotmail.com wrote:
> Hi Mr.David,
>
> Congratulations for the great job!
>
> Could you point me to OMI and MWAPI documentation that you intend to implement on RSM?
>
> Thank you very much! =)
>
> Regards.

Sure. The OMI standard is at https://www.iso.org/obp/ui/#iso:std:iso-iec:15851:ed-1:v1:en and
the MWAPI standard is at https://www.iso.org/obp/ui/#iso:std:iso-iec:15852:ed-1:v1:en. I'm not
sure how much they cost right now, but you might be able to find copies that are cheap or free if
you look, but I don't know where. Thank you.

[ed de moel]

You could look at http://71.174.62.16/Demo/AnnoStd
You'd have to scroll down quite a bit through the menu to get to OMI and MWAPI,
but the complete standards are there.

Shortcuts are:
OMI: http://71.174.62.16/Demo/AnnoStd?Frame=Main&Page=b100001
MWAPI: http://71.174.62.16/Demo/AnnoStd?Frame=Main&Page=f000001

[David Wicksell]

Thank you Ed! That's very handy.

[Flávio Fornazier]

Thank you for your reply Mr.David,

Now I can get ahead to try to be helpful for future testing.

Regards.

[ed de moel]

There's always http://71.174.62.16/Demo/AnnoStd
It has the standards "page by page", and you'd have to scroll down the
menu quite a bit to get to OMI and MWAPI, but the complete standards are there.

Shortcuts:

[Flávio Fornazier]

That's great! As a regular visitor since early's 2000s, I guess the only thing left to say is many thanks to Mr.Ed de Moel for keep the website available and free of charge for so long.