A Caché of Tips - Using $lists

308 views
Skip to first unread message

Ed Clark

unread,
Jan 31, 2012, 9:50:41 AM1/31/12
to InterSy...@googlegroups.com
Caché Object Script provides support for a specially formatted string called a $list. A $list is used in object script in many of the places where a dynamic array would be used in mv basic. As with a dynamic array, elements can be added, updated, or removed at any location in a $list. Unlike a dynamic array, a $list can contain any character--elements in a $list are stored in a binary structure rather than being separated by system delimiters. This structure also makes accessing arbitrary elements faster than in a dynamic array. $Lists are used extensively by Caché Objects and in the object library, so intrinsic support for $lists has been added to the mvbasic language.

A $list is usually created using the $LISTBUILD() function, and accessed with the $LIST() function:

USER:;L=$LISTBUILD("A","B",1,"2")
USER:;CRT $LIST(L,2)
B

A $list is stored as a string, so it could be displayed and manipulated as a string, but in general should not be because it contains binary characters. A $list should not be stored in a dynamic array or in an mv file because it could contain system delimiters.

If you are curious what is actually inside of a $list:

USER:;CRT OCONV(L,'MCP')
..A..B.....2
USER:;CRT OCONV(L,'MCAX')
030141030142030401030132

The MCP conversion displays non-printable characters as dots, and the MCAX conversion displays the ascii values of characters as 2 hex digits. Displaying the $list unconverted with a CRT statement could have bad results if it contains binary characters that a terminal recognizes as control strings. However, the Object Script ZWRITE command knows how to display a $list:

USER:; ^list=L
USER:; $xecute("zwrite ^list")
^list=$lb("A","B",1,"2")

And you can also display the value of $lists in objects:

USER:;s="%ListOfDataTypes"->%New()
USER:;s->Insert($LISTBUILD("ABC",1,"2"))
USER:;$SYSTEM.OBJ->Dump( s)
+----------------- general information ---------------
| oref value: 3
| class name: %Library.ListOfDataTypes reference count: 0
+----------------- attribute values ------------------
| Data(1) = $lb("ABC",1,"2")
| ElementType = ""
| Size = 1 <Set>

However, the actual format of a $list is not documented, and you shouldn't go inside. Usually, a $list is only used with a set of functions that know how to handle a $list. These functions are intrinsic in both Object Script and Mv Basic:

$LIST --retrieve elements from a list
$LISTBUILD -- create a list
$LISTFROMSTRING --create a list from a delimited string
$LISTTOSTRING -- create a delimited string from a list
$LISTFIND -- find a string in a list
$LISTNEXT -- traverse a list in order
$LISTLENGTH -- get the number of elements in a list
$LISTSAME -- compare lists
$LISTVALID -- determine if a string contains a list
$LISTDATA -- determine if a list contains data in an element
$LISTGET -- retrieve elements from a list that may contain empty elements

A $list can be efficiently traversed using $LISTNEXT():

L=$LISTBUILD("A","B","C","D")
CNT=0
LOOP WHILE $LISTNEXT(L,CNT,ELEMENT) # 0 DO
CRT ELEMENT
REPEAT

Which will print out all the elements of the list. $$LISTNEXT() is much faster than using $LISTLENGTH() and a for/next loop.

A $list can contain other $lists as elements, which in turn can contain other $lists. A $list can also contain null elements, which are different from empty elements (similar to the SQL NULL concept):

USER:; L1=$LISTBUILD("A","","B")
USER:; L2=$LISTBUILD("A",,"B")
USER:;CRT $LIST(L1,2)

USER:;CRT $LIST(L2,2)
<NULL VALUE>

In L1, the second element is an empty string, but in L2 the second element is null. Accessing a null element with $LIST() will throw a <NULL VALUE> exception. You can use $LISTGET() to handle possibly null elements:

USER:;CRT $LISTGET(L2,2,"i am null")
i am null

And you can use $LISTDATA() to determine if an element is null:

USER:;CRT $LISTDATA(L2,2)
0
USER:;CRT $LISTDATA(L1,2)
1

If a list contains only 3 elements, and you ask $LIST() to return element #4, you also get a <NULL VALUE> exception.

$LISTVALID(s) returns true if string s is a $list:

USER:;crt $LISTVALID("abc")
0

An empty string "" *is* a valid list, containing no elements.

$lists can be concatenated with each other forming a larger list:

(using L, L1, and L2 defined in previous examples) USER:; L3=L:L1:L2 USER:; CRT $LISTLENGTH(L3) 10

$lists remember the data type of elements they are built with:

USER:;^L=$LISTBUILD("AB4C","1",2,FDIV(9,3))
USER:[ZW ^L
^L=$lb("AB4C","1",2,3)

Note that in the zwrite output, "1" is in quotes because it was passed as a string. 2 doesn't have quotes because it was passed as a number. The 4th element 3 also displays as a number, but it is actually stored in the $list as a floating point number.

The object library provides a datatype class %Library.List, or %List to represent $lists.

Bill Farrell

unread,
Jan 31, 2012, 11:16:16 AM1/31/12
to InterSy...@googlegroups.com
This is a bit of synchronicity.  A couple of days ago I was struggling with lists.  I had a property in a class that was defined as a list.  While I was debugging in the list, the contents of the property (and the code) behaved perfectly as expected.  When I returned to the routine, I could find no way to view that property as a list still.  I tried making an instance of a list then setting the value of the instance to the value of the property of the instance with the list property.  I thought if XYZ->Property was a list and in my routine I set Lst = XYZ->Property, it should "just go".

Uh uh.  Then I tried making Lst an instance of a list.  Still no values.  Exception, exception, exception.  Frustration.

Is there a special trick for passing list-type properties to routines?

I did try ListOfDatatypes.  I tried collections.  Nothing but a md array seemed to get me close to what I wanted, but it still isn't elegant.  I could insert my datatype object into the list and I could see what I expected to see inside the class code.  When I tried to access an element of datatype in the list in the routine, the entire variable was either empty, unassigned, or if it wasn't empty, caused an exception when I tried to access an element.

In VB or PHP, such a thing is easy to make.  An instance of a datatype class in this case is defined as an object that has two properties on its own.  In the main class, an element in the array is an instance of a datatype.  There would be one dimension in the containing array/list/whatever-object, being an ordinal count from [0,1]...n .  Thus the mini-objects I create in the main class stay in a regimented order.  This would also give me the ability to know an exact count of elements at any given moment and the ability to either iterate neatly from 1 to n or dip in the middle of the array/list/whatever-object and pull out one particular element which should be an instance of my datatype.

I'm sure there's a way to accomplish the same thing in Cache.  I just haven't found the way of going about it.

Speaking of md arrays, I noticed a trick in one of the system-supplied classes for keeping a count of the first dimension.  This is probably old hat to more experienced Cache programmers but it didn't click to me at first that md arrays are complex types.  This isn't quite clear in the docs (which in the MV doc should state (1) what a complex type is (most MV programmers don't deal with those much) and (2) how to leverage the beauty a complex type can offer.  As I was trying to figure out how to accomplish the list of custom-datatype I was poring through Cache code in the Class Browser, I found a helpful tidbit in COS that I leveraged in MVBasic:

Dim SomeArray() ; * grab hold of a multidimensional array
SomeArray = 0 ; * a node can contain data and a downward pointer, so keep count here
SomeArray += 1 ; SomeArray( SomeArray ) = "some string to keep around"

So long as the element value is a simple, this works great... and Cache arrays are blinding fast to work with as opposed to MV dynamic arrays.   It works lovely for long data-gathering and shaping loops.

The syntax is a bit clumsy as compared to some other languages, but since there is no count() function for arrays (say, like PHP has) to give you a number of elements, it's a workable workaround.  ( understand now why a count() function may be impractical, since md arrays are a complex type.  If there were a count() function, what would one be counting, a count of the desired dimension or a count of the count of downward nodes?  How many levels down would one count.  Not practical.  Now it makes sense.

Still in all, I'd like to know how to make an enumerated array of a custom datatype and how to pass such an object back to a routine intact.

Thanks,
B
Caché Object Script provides support for a specially formatted string called a $list. A $list is used in object script in many of the places where a dynamic array would be used in mv basic. As with a dynamic array, elements can be added, updated, or removed at any location in a $list. Unlike a dynamic array, a $list can contain any character--elements in a $list are stored in a binary structure rather than being separated by system delimiters. This structure also makes accessing arbitrary elements faster than in a dynamic array. $Lists are used extensively by Caché Objects and in the object library, so intrinsic support for $lists has been added to the mvbasic language.

A $list is usually created using the $LISTBUILD() function, and accessed with the $LIST() function:

USER:;L=$LISTBUILD("A","B",1,"2")
USER:;CRT $LIST(L,2)
B

A $list is stored as a string, so it could be displayed and manipulated as a string, but in general should not be because it contains binary characters. A $list should not be stored in a dynamic array or in an mv file because it could contain system delimiters. 

If you are curious what is actually inside of a $list:

USER:;CRT OCONV(L,'MCP')
..A..B.....2
USER:;CRT OCONV(L,'MCAX')
030141030142030401030132

The MCP conversion displays non-printable characters as dots, and the MCAX conversion displays the ascii values of characters as 2 hex digits. Displaying the $list unconverted with a CRT statement could have bad results if it contains binary characters that a terminal recognizes as control strings. However, the Object Script ZWRITE command knows how to display a $list:

USER:; ^list=L
USER:; $xecute("zwrite ^list")
^list=$lb("A","B",1,"2")

And you can also display the value of $lists in objects:

USER:;s="%ListOfDataTypes"->%New()
USER:;s->Insert($LISTBUILD("ABC",1,"2"))
USER:;$SYSTEM.OBJ->Dump( s)
+----------------- general information ---------------
|      oref value: 3
|      class name: %Library.ListOfDataTypes reference count: 0
+----------------- attribute values ------------------
|            Data(1) = $lb("ABC",1,"2")
|        ElementType = ""
|               Size = 1  <Set>

However, the actual format of a $list is not documented, and you shouldn't go inside. Usually, a $list is only used with a set of functions that know how to handle a $list. These functions are intrinsic in both Object Script and Mv Basic:

  $LIST --retrieve elements from a list
  $LISTBUILD -- create a list
  $LISTFROMSTRING --create a list from a delimited string
  $LISTTOSTRING -- create a delimited string from a list
  $LISTFIND -- find a string in a list
  $LISTNEXT -- traverse a list in order
  $LISTLENGTH -- get the number of elements in a list
  $LISTSAME -- compare lists
  $LISTVALID -- determine if a string contains a list
  $LISTDATA -- determine if a list contains data in an element
  $LISTGET -- retrieve elements from a list that may contain empty elements

A $list can be efficiently traversed using $LISTNEXT():

L=$LISTBUILD("A","B","C","D")
CNT=0
LOOP WHILE $LISTNEXT(L,CNT,ELEMENT) # 0 DO
  CRT ELEMENT
REPEAT

Which will print out all the elements of the list. $$LISTNEXT() is much faster than using $LISTLENGTH() and a for/next loop.

A $list can contain other $lists as elements, which in turn can contain other $lists. A $list can also contain null elements, which are different from empty elements (similar to the SQL NULL concept):

USER:; L1=$LISTBUILD("A","","B")
USER:; L2=$LISTBUILD("A",,"B")
USER:;CRT $LIST(L1,2)

USER:;CRT $LIST(L2,2)
<NULL VALUE>

In L1, the second element is an empty string, but in L2 the second element is null. Accessing a null element with $LIST() will throw a <NULL VALUE> exception. You can use $LISTGET() to handle possibly null elements:

USER:;CRT $LISTGET(L2,2,"i am null")
i am null

And you can use $LISTDATA() to determine if an element is null:

USER:;CRT $LISTDATA(L2,2)
0
USER:;CRT $LISTDATA(L1,2)
1

If a list contains only 3 elements, and you ask $LIST() to return element #4, you also get a <NULL VALUE> exception.

$LISTVALID(s) returns true if string s is a $list:

USER:;crt $LISTVALID("abc")
0

An empty string "" *is* a valid list, containing no elements.

$lists can be concatenated with each other forming a larger list:

(using L, L1, and L2 defined in previous examples) USER:; L3=L:L1:L2 USER:; CRT $LISTLENGTH(L3) 10

$lists remember the data type of elements they are built with:

USER:;^L=$LISTBUILD("AB4C","1",2,FDIV(9,3))
USER:[ZW ^L
^L=$lb("AB4C","1",2,3)

Note that in the zwrite output, "1" is in quotes because it was passed as a string. 2 doesn't have quotes because it was passed as a number. The 4th element 3 also displays as a number, but it is actually stored in the $list as a floating point number. 

The object library provides a datatype class %Library.List, or %List to represent $lists.

-- 
You received this message because you are subscribed to the Google Groups "InterSystems:  MV Community" group.
To post to this group, send email to Cac...@googlegroups.com
To unsubscribe from this group, send email to CacheMV-u...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/CacheMV?hl=en

Gandalf

unread,
Jan 31, 2012, 11:54:34 AM1/31/12
to InterSystems: MV Community
Very good information!

A couple points:

You can not store orefs in a $LIST (just as you cannot store them in a
global node).
If you try, it will just store the printed representation...

Cache (and Cache MV Basic) is (mostly) a typeless language... (most)
everything is always considered to be a string.
(There are really 3 basic types now, strings, orefs, and IEEE 64-bit
binary floating point #s)

Internally, *as an optimization*, we can also store values as 32-bit
signed integers, and 64-bit integers scaled by a 1-byte decimal
exponent (from 10**-128 to
10**127), however, you should not think of those as being real "data
types"... people get into trouble with that all the time, because
Cache can convert from
one internal type to another, it doesn't guarantee it will preserve a
value as one "type" or another...
This is probably less of a problem for MultiValue programmers, as they
are used to #s being converted to string format automatically when
placed into a dynamic array, and doing arithmetic on string
representations of numbers directly...

Since a list is just an arbitrary string, $LISTVALID can't tell you if
something was really *meant* to be a $LIST, rather,
it just tells you whether it is not a correctly formed list.
This can come up if you have a short string with control characters in
it... the string $C(1,1,1) (three bytes of 0x01), is a valid $list,
with 3 elements,
each of which is empty, but some customers use $C(1) as a delimiter...
so that *could* have been a delimited string with 4 elements separated
by $c(1).

You can also look quickly at the contents of a $LIST by doing: CRT
$LISTTOSTRING(lst)

Ed Clark

unread,
Jan 31, 2012, 11:55:26 AM1/31/12
to intersy...@googlegroups.com
Do you have an example of what you were trying to do that doesn't work?

Bill Farrell

unread,
Jan 31, 2012, 12:33:16 PM1/31/12
to intersy...@googlegroups.com
Ah, ok, then it makes perfect sense if any type can be converted automatically instead of certain types.  I've always joked that strong typing is for weak minds (h/t Stan Kelly Bootle, "The Devil's DP Dictionary") which make MV products so easy to use.  Strongly-typed languages like VB or ones that are loosely-typed like PHP have their own quirks.  I wrongly assumed that if one could create a certain datatype in Cache (which, kindasorta you can), the system would respect that type as its own kind of thing rather than trying to convert it.

As it is, using a multidimensional array works well enough.  MD arrays are beautifully fast and I've come to use them much in my MV code. "Classic" MV bogs down trying to manage long dynamic arrays and in many versions, once a DIMensioned array was declared, you couldn't REDIM it (INFO lets you do this, but I seem to be a rarity, preferring INFO to all other flavors).

A fair compromise has been reached in my mind, even if it's not the structure I'm idealizing.  I'm declaring success and marching on.

Thanks for explaining it.

Bill
Reply all
Reply to author
Forward
0 new messages