UserRPL Matrix Evaluation functions
John H Meyers
[New thread for a separate product :]
Have you noticed that the Matrix Editor in "list of lists" mode
(flag -91 set) is incapable of editing a 1D list or vector?
When you edit a 1D vector or list, upon completion with ENTER
it always produces a 2D result (in "list of lists" form);
it is impossible, in fact, to output a 1D list, since
pressing the VEC menu key to ensure that it has a "dot"
is simply ignored.
This means, for example, that if you extracted a column
of an original array and left it as a simple list or vector,
any editing would turn that original "column" back into
a 1-ROW matrix, not what was originally extracted,
and you would then have to use TRAN to make it a column again.
Note also that when you replace part of an array
using the built-in REPL command,
you can not tell that command any "range" of original
cells to replace -- you can tell it only the location
of the *first* cell to replace, and the remainder
are then completely determined by the size and shape
of the replacement matrix.
So in the UserRPL function set below, we go with the way
the calculator works with matrices -- when extracting any
*range* of cells, e.g. 'Xmm-Ynn' (which could turn out to be
either horizontal, vertical, or a rectangle), we are entirely
consistent with the calc, and *always* return a two-dimensional
array, so that subsequent editing by the Matrix Writer
can not change its form, and so that the shape and orientation
is *always* remembered, although you can change it
via TRAN or via ARRY\-> DROP { newrows newcols } \->ARRY
The function set below therefore consists of the following:
MSGET - Extract either a range (sub-matrix) or a single element.
MSPUT - Replace either a single element or a sub-matrix,
depending upon the type of replacement object.
MSEVAL - Evaluate entire matrix,
interpreting 'A3' etc. as "cell variables,"
and allowing non-algebraic objects in cells,
using a *static*copy* of the original array as basis.
MSEXPR - Evaluate an expression, e.g. 'C1+B2'
(based on matrix last stored by MSEVAL)
Auxiliary utility functions:
LXA - Like AXL but permits non-algebraic objects in Matrix;
sets flag -91 when an "illegal" matrix is created.
MCOORD - Converts cell position 'Y12' <-> { 12 25 }
MRANGE - Converts range 'A3-B1' <-> { 3 1 } { 1 2 }
MSGENR - Incremental cell formula generator [more info to come];
follow with { rows cols } \->ARRY to produce desired
dimensions, then 'pos' MSPUT to insert into array.
MPURGD - Purge the time-saving temporary work area.
Cell ranges can be expressed like 'A3-C2'
(specifying any two corners, in any order),
individual cell positions as your choice of
'C5' or { row col } or as the ordinal element number
(the latter two forms are as used anyway
by the built-in GET, PUT, SUB and REPL commands).
Results are always delivered in the same form
(Matrix or "list of lists") as was the input;
use LXA to convert {{1 2}{3 4}} <-> [[1 2][3 4]]
Currently limited to 26 columns (A-Z),
unlimited rows (until memory is full :)
@ Purge temp. directory
\<< 'M.dir' PGDIR \>> 'MPURGD' STO
@ {{1 2}{3 "4"}} <-> [[1 2][3 "4"]] [allowing any objects]
\<< -55. CF IFERR AXL THEN -91. SF
1. \<< DUP TYPE 5. == { #16A006h FLASHEVAL } IFT
\>> DOSUBS #16A006h FLASHEVAL END \>> 'LXA' STO
@ 'A3-B1' <-> { 3 1 } { 1 2 }
\<< IF DUP TYPE 9. == THEN \->LST 1. 2. SUB
1. 'MCOORD' DOLIST LIST\-> DROP ELSE 2. \->LIST
1. 'MCOORD' DOLIST { - } + \->ALG END \>> 'MRANGE' STO
@ 'Y12' <-> { 12 25 }
\<< DUP TYPE 6. == {
S~N DUP TAIL STR\-> I\->R SWAP NUM 64. - 2. \->LIST } {
LIST\-> DROP 64. + CHR SWAP R\->I \->STR + S~N } IFTE \>>
'MCOORD' STO
@ [[M]] object 'A3' (overlay one cell or 2D matrix)
@ [[M]] object pos (overlay one cell or 2D matrix)
\<< DUP TYPE 6. == { MCOORD } IFT
SWAP {3. 4. 29.} OVER TYPE POS NOT
{ 1. \->LIST LXA 1. \->LIST LXA } IFT
ROT DUP TYPE 5. == DUP 5. ROLLD { LXA } IFT
UNROT REPL SWAP { AXL } IFT \>> 'MSPUT' STO
@ [[M]] 'B2-C3' -> [[m]] (submatrix, always 2D)
@ [[M]] 'A3' -> object (one cell's content)
@ [[M]] pos -> object (one cell's content)
\<< SWAP DUP TYPE 5. == UNROT PICK3 { LXA } IFT SWAP
DUP TYPE 9. == { MRANGE SUB SWAP { AXL } IFT }
{ DUP TYPE 6. == { MCOORD } IFT GET NIP } IFTE \>>
'MSGET' STO
@ 'C1+B2' EVAL -> ??? (requires prior MSEVAL)
\<< M.dir EVAL UPDIR \>> 'MSEXPR' STO
@ [[...]] EVAL -> [[...]]
\<< -55. CF DUP TYPE 5. == SWAP OVER { LXA } IFT
'M.dir' DUP IFERR CRDIR THEN DROP END EVAL DUP SIZE
LIST\-> DROP \-> r c \<< 1. r FOR r 1. c FOR c
r c 2. \->LIST DUP2 GET SWAP MCOORD STO NEXT NEXT
1. r FOR r 1. c FOR c r c 2. \->LIST DUP2 GET
WHILE DUP TYPE 6. == REPEAT RCL END EVAL @ must exist
1. \->LIST #16A006h FLASHEVAL 1. \->LIST #16A006h FLASHEVAL
REPL NEXT NEXT UPDIR SWAP { AXL } IFT \>> \>>
'MSEVAL' STO
@ Example (no need for 'A2+0' in cell B1):
{{'B1+B2' A2 4 "ABC" }
{ '1+2' 5 'A1+B1+C1' D1 }} DUP MSEVAL
@ [End]
manjo
> PS: What is that
> #16A006h FLASHEVAL
Hello,
#16A006h is ^{}TO[]
(CAS thing for list->matrix)
:-)
#17A006h ^LIST2MATRIX
may be safer in some cases
There is a good reason why TMP directory should be created and destoryed at
run-time.
Example: when linking several tables
-while working with one table and going in to the other (while linking
evaluated cells -cells containing a formula) it could represent a problem.
(you can't use one directory to eval linked tables)
also writing this kind of program in userRPL...is questionable (not that i
don't like userRPL)
At some stages i deliberately use userRPL because i don't bother doing
in-depth checks, using userRPL at
convenient places makes sure that program would break in a friendly manner
if arguments are of unexpected value.
userRPL has way too much overhead, so i try to avoid it as much as possible
in serious work.
By serious work i mean when i'm doing something dedicated.
On the other hand userRPL is great for small ad-hoc qick and dirty programs
:-)
Also sysRPL code is quite different from userRPL : most commands have
different order of arguments, sometimes even number of arguments or results
returned is different...
also prefered data type is BINT rather than real or zint. (that's something
we're aware of -right ?)
see you later,
manjo
P.S.
at this time i'm playing with matrices
-thinkin about what could be next usefull addition to the library.
manjo
"47A20" "68620" ROT ::SWAP
IFT SREPL DROP H->
>>
Nice thinking...
-i was thinking the same way (for a short time)
-discarded it (luckily in time) because even with remote possibility 47A20
may come as data (within string or whatever),
-it would be mistaken for a prologue of course it would get replaced with
68620 prologue giving unexpected EXTREMLY heavy to locate bug :-)
My 1st concern is to make it work, only 2nd to make it safe :-)
manjo
Veli-Pekka Nousiainen
> << DUP ->H SWAP TYPE 5. =/ @ not equal
> "47A20" "68620" ROT ::SWAP
> IFT SREPL DROP H->
>>>
>
> Nice thinking...
> -i was thinking the same way (for a short time)
> -discarded it (luckily in time) because even with remote possibility
> 47A20 may come as data (within string or whatever),
AAAARRRRGGGGHHH!
> -it would be mistaken for a prologue of course it would get replaced
> with 68620 prologue giving unexpected EXTREMLY heavy to locate bug :-)
>
> My 1st concern is to make it work, only 2nd to make it safe :-)
> manjo
Huh?!
Yours is safer than mine?!
(is it safer than John's approach?)
John H Meyers
> #16A006h is ^{}TO[]
> (CAS thing for list->matrix)
Although this FPTR is in the CAS part of ROM,
it simply does CHANGETYPE;
this is also why it must be done recursively
(first on any inner lists and then again on outer list).
By the way, LXA is nothing new -- here it was back in '02:
http://groups.google.com/group/comp.sys.hp48/msg/5f83e1c01ad5360f
> #17A006h ^LIST2MATRIX
> may be safer in some cases
That's what AXL does; fails on {{"ABC"}}
(and thus on any spreadsheet containing strings),
which is the very reason why LXA is needed.
You'll note that the current LXA first tries AXL,
and only forces the result (along with setting flag -91)
if AXL (and the above entry point) can't be used;
the UserRPL application as a whole now also
returns results in the same form (matrix or list)
as supplied by the user, even if an intermediate "matrix"
was utilized for simpler and more efficient programming
(the UserRPL version is basically built around a set of
more capable array commands, which don't work with the
"list" form).
> There is a good reason why TMP directory should be created
> and destroyed at run-time.
> Example: when linking several tables
Nothing can be lost by leaving a temp directory unpurged,
although when only one temp directory is ever created at one time,
it will hold only the most recently evaluated array.
I found that execution time is greatly reduced upon re-using
the temp directory, so I decided to take advantage of that.
Of course, different folders can each have a temp directory,
and there's no reason why multiple temp directories can't
be used even in one place,
if later expansion suggests this to be useful.
> also writing this kind of program in userRPL...is questionable...
> userRPL has way too much overhead
That's way too over generalized (and prejudiced) a statement.
There are advantages to each approach; when UserRPL
has all the tools needed, it's perfectly satisfactory to me,
including its safety, as well as often being close enough
in timing that a SysRPL version isn't worth my while.
> so i try to avoid it as much as possible in serious work.
A lot of serious work leads to serious crashes when
attacking easy problems entirely in SysRPL
and leaving something unprotected;
I have no objection to SysRPL, but some users
can more easily modify UserRPL for themselves,
and I personally prefer to use for myself
the most accessible tool that does a satisfactory (and safe) job,
particularly on a first implementation.
> On the other hand userRPL is great for small ad-hoc
> quick and dirty programs :-)
People might be surprised to find UserRPL employed in ROM, then
(the SEQ command is implemented in UserRPL, for example).
> Also sysRPL code is quite different from userRPL...
So? I've done SysRPL and UserRPL programming, and for each new task,
I choose what I think can do the job, favoring the easiest, safest,
and fastest way to develop it myself.
It seems like a good idea to have a choice of many different
programs and approaches, however, and may all enjoy the variety.
[r->] [OFF]
John H Meyers
> \<< DUP ->H SWAP TYPE 5. \=/ @ not equal
> "47A20" "68620" ROT ::SWAP
> IFT SREPL DROP H\-> \>>
I don't see what post introduced this;
it seems to be an attempt to perform CHANGETYPE
(in bulk, on an entire set of lists of lists?)
and is flawed, as Manjo has just noted
(it would also even be applied to an array element
which was a list, or to a list within a program
in an array element, for yet more flaws).
But that's how huge catches of fish are now caught;
so what if there are a few dolphins, old tires,
and perhaps a sailor's bones winding up in the same can,
as long as it's *mostly* what was intended to be caught :)
[r->] [OFF]
manjo
i agree for the most part.
However if you consider that sysRPL can run 2-3 even up to 10 times faster
than userRPL (for several obvious reasons)
... it may as well end up being faster even without persistent temp dir.
-a couple of times i started something with userRPL then it would turn out
with sysRPL would be
smaller and faster (sometimes even a single command in sysrpl) and then (to
write it in sysRPL i had to start all over again :-)
keep up the good work, see you later...
manjo
Veli-Pekka Nousiainen
> On Sun, 29 Oct 2006 17:54:24 -0600, manjo wrote:
Are you going to support the build-in spreadsheet on the new ROM 2.10-7
???
external programs are still useful
As would be examples in ENGLISH
a new Quick Guide dedicated to the build-in spreadsheet is needed
even if it would doublicate any old VisiCalc Manual
Matrix is here to stay!
John H Meyers
if the evaluation program recursively calls itself,
the "inner" evaluation will automatically use
a different temp subdirectory (within the first one),
so nothing more need be done to accommodate that issue.
Real life work puts play on hold, however, so more later...
[r->] [OFF]
Veli-Pekka Nousiainen
So linked matrices propose only memory allocation problems
but work beautifully when recursion is used
??
manjo
> > with 68620 prologue giving unexpected EXTREMLY heavy to locate bug :-)
> >
> > My 1st concern is to make it work, only 2nd to make it safe :-)
> > manjo
>
> Huh?!
>
> Yours is safer than mine?!
> (is it safer than John's approach?)
He he,
This shows we have similar ideas :-)
I think John's approach is safer, if you consider userRPL safer than sysRPL
however :
(the use of 'dirty' matrices is not entirely supported by built in userRPL
functions...
-that's why MXE doesn't convert back to matrices when it's possible that it
will be wicked :-)
manjo
John H Meyers
> the use of 'dirty' matrices
> [that is, containing non-algebraic items in some cells]
> is not entirely supported by built in userRPL functions...
I get an ordinary "Bad object type" in any case
of trying to do algebraic operations as an experiment.
The reason for LXA is, again, that it's the *only*
way to get Matrix-oriented GET, SUB, REPL, etc. to work.
> -that's why MXE doesn't convert back to matrices
> when it's possible that it will be wicked :-)
And the UserRPL programs return the same type as the user
supplies, regardless of using any intermediate array
for GET, SUB, and REPL -- so you'll never get
a "non-algebraic" result back from them unless
*you* have forced it into that form to begin with.
In addition, flag -91 gets automatically set
if any such item is detected in list format,
so that Matrix Writer will also always return lists,
even if given a non-algebraic "array" to edit.
You are invited to try causing a problem with this;
let me know if you can.
[r->] [OFF]
manjo
it would break with error.
it seamed logical to me to convert to lists which is more flexible for
various data-types
thats all :-)
> The reason for LXA is, again, that it's the *only*
> way to get Matrix-oriented GET, SUB, REPL, etc. to work.
yes, i understand this as well but still
The point of using standard data format and tools ?
anyway as it seems
both our experiments will soon become obsolete :-)
still ... keep up the good work !
manjo
Veli-Pekka Nousiainen
I din't think so
The Fro..Frenchmen are not stickinh together
That is
I don't believe that Bernards new Spreadsheet
will be accepted by Cyrille (or his boss)
already Jean-Yves has hinted that it's not going to go into the ROM
Also note that Cyrille has unofficially promised to patch
all the future ROMs with HPGCC library
so that C_applications will run from stack etc
Maybe they are CODE objects?
DO we "lose" yeat another 128 KB bank of Flash
if the ROM is upgraded to accommodete all the goodies?
If Bill Gates & MS-DOS fits into 640 KB
so will my libraries
I surely hoped that 50G would have incorporated a bigger Flash
Think About 4MB with 2MB for OS
Well - maybe we get a HP-51G (and 90Mhz speed)
which has an optional Lion cell battery
which can be loaded via USB
John H Meyers
> it seemed logical to me to convert to lists
> which is more flexible for various data-types
While it seemed logical to me to *accept* lists as input
and *return* lists as results if so, for the very same reason
you give -- because you can insert anything into them --
but it's no concern of the user what goes on in between,
so that anything which makes that in-between processing
more efficient (and which always works) seems fine to me.
It also seems fine to me to be equally *able*
to use a regular array as input and output,
*if* it suits both the user and the calculator,
so what's wrong if I do? (so could you, if you like!)
Thus far, however, the words "convert lists <-> arrays"
may have some readers imagining some big operation going on,
so let's delve a bit more into what these things consist of,
which will reveal that there is really *no* conversion at all,
just a "labeling change," in effect,
comparable to re-branding a cheaper Kinpo calculator
(perhaps itself only a rip-off of some Casio :)
by merely sticking an "HP" logo on it :)
Let's just look at the "list of lists" form
vs. the "type 29 array" form, e.g.
{{ 'A1' 'B1' }{ 'A2' 'B2' }}
then after AXL...
[[ 'A1' 'B1' ][ 'A2' 'B2' ]]
These objects are actually *identical* in memory
except that the object prolog (first 20 bits or 5 hex nibbles)
of each list has been changed to a different ROM address
(try \->H and carefully compare the 74-nibble hex strings);
AXL does nothing more in changing "lists" to an "array,"
and this is also the identical thing that LXA does,
using the same internal "flashptr" function that AXL uses.
So in fact there is no difference in substance here,
*but* there are very neat array-oriented UserRPL
commands (GET, PUT, SUB and REPL) which all
can use the { row col } "array position" indicators,
and both SUB and REPL can extract or replace
rectangular sub-arrays, while *none* of those UserRPL commands
works so powerfully with the form labeled as "{{...}}" --
these comands only do their "array magic" upon the form labeled
as "[[...]]", so into that form I'll *temporarily* make any input,
even if I'll give it back later in whatever form
matches whatever the user originally supplied.
All the more difficult intermediate work is then relegated
to the built-in smarts of the GET, SUB and REPL commands,
which all *don't*care* what's inside those "arrays,"
because they are still actually lists anyway,
and can contain anything we like.
For anyone who wants to implement all internal processing
*directly* upon the "list of lists" form,
duplicating the already built-in "array smarts" of those UserRPL
commands, hats off to you for doing all that extra work :)
The built in viewer (TOOL VIEW, which does DUP SCROLL)
also presents a much nicer-looking view for type 29
than for lists, so you are even free use that form
for a nicer display, if the mood grabs you :)
The Matrix writer will open either one of these forms
at any time, no matter what is in it,
and with flag -91 set (which LXA sets for you
any time that a non-algebraic cell value is found
while changing lists to type 29),
the Matrix Writer also always returns back a "lists" form,
even if you feed it a type 29 to begin with,
so there's absolutely nothing you can't do with a spreadsheet,
whichever way you care to go -- your library can handle anything,
and so can UserRPL functions handle anything,
with the neat assistance of some "smart" UserRPL commands
(acting upon an intermediate copy of the original input)
to spare the need for quite as much programming.
Another aside:
When we inspect each UserRPL command to see what it does,
it turns out that we find a bunch of SysRPL commands --
actually, each UserRPL command is the *same* as a SysRPL
command, except that its *first* internal SysRPL command
makes sure that the stack has the right input objects
on it (enough objects and acceptable types), and remembers
the arguments, command name and stack depth for error backout.
When the set of UserRPL commands just happens to very well fit
the task at hand, then, we can often make a UserRPL program
that is simpler than a "pure SysRPL program" for the same task.
Indeed, because certain internal UserRPL functions which implement
some UserRPL commands aren't even "supported" (may change locations
in different ROM versions), one sometimes *has* to embed UserRPL
commands right in the middle of a SysRPL program, usually adding
two extra commands (quote and EvalNoCK) to bypass argument
processing (which in the SysRPL program is usually up-front instead).
Whatever the comparison between final processing times,
it isn't merely "SysRPL vs. UserRPL" which matters;
it's also a matter of whether UserRPL "array processing"
is done as efficiently, and also whether keeping
a temporary directory of *original* input cells
happens to save time or not, and also which calculator
all this runs on, then finally the apples will be either
faster or slower than the oranges -- but the programs
may still be different in size and complexity.
I've stopped working further on this UserRPL stuff
for the moment (tho for "linked arrays" I have some ideas);
Manjo is finished, so there's a completed library
to use anyway, and whatever may be in the works
for HP (if anything) looks like yet another different
stylistic direction, but that's the good old
smorgasbord that the world is anyway,
with so many differently prepared dishes to choose from :)
One or two people might have learned more about "arrays"
(and even about programming) in the meantime,
by wading through all this otherwise uninspiring stuff;
hope it was of interest.
Best wishes from http://www.mum.edu
and http://www.maharishischooliowa.org
Veli-Pekka Nousiainen
> UserRPL Matrix Evaluation functions
>
> [New thread for a separate product :]
When you are both "finished" with your work
maybe you could join forces
X
> @ Purge temp. directory
> \<< 'M.dir' PGDIR \>> 'MPURGD' STO
Nice thing that temp.dir
> @ {{1 2}{3 "4"}} <-> [[1 2][3 "4"]] [allowing any objects]
> \<< -55. CF IFERR AXL THEN -91. SF
> 1. \<< DUP TYPE 5. == { #16A006h FLASHEVAL } IFT
> \>> DOSUBS #16A006h FLASHEVAL END \>> 'LXA' STO
===============================================
How about something like
<< DUP ->H SWAP TYPE 5. =/ @ not equal
"47A20" "68620" ROT ::SWAP
IFT SREPL DROP H->
>>
--
Veli-Pekka Nousiainen
X
> I've stopped working further on this UserRPL stuff
BWAAAAH! :´-( sniff...
> for the moment (tho for "linked arrays" I have some ideas);
> Manjo is finished, so there's a completed library
> to use anyway, and whatever may be in the works
> for HP (if anything) looks like yet another different
> stylistic direction, but that's the good old
> smorgasbord that the world is anyway,
> with so many differently prepared dishes to choose from :)
>
> One or two people might have learned more about "arrays"
> (and even about programming) in the meantime,
> by wading through all this otherwise uninspiring stuff;
> hope it was of interest.
more, please!
John H Meyers
> more [UserRPL programs], please!
You've written over 5000 articles in comp.sys.hp48,
so come on -- I bet you already know UserRPL yourself :)
[r->] [OFF]
Veli-Pekka Nousiainen
yes but your ideas about MLINK keeps me tickling....
I just can't wait - this whole spreadsheet thingy got me overboard^oo