Dynamic Arrays and Cache Arrays

[Bill Farrell]

In the MV world, our notion of array doesn't always fit into the Javascript/Python/PHP notion of arrays. Personally, I dislike writing loops to compare dynamic array, for instance. It would be so nice if there were an array structure that could do things like diff, or intersect, or a PHP-like shift,unshift, push, or pop.

There is now.

One case that saved me a lot of coding and frustration is where in EasyCSP, a user's group memberships are a multivalued attribute. The access rules are Cache' %Lists. What's the easiest way of comparing the two to see if any of the user's groups are in the access rule "allow" list.

To wit:

    // get the list of groups to which this user belongs
    set myGroups = myself.getAttributeValue("groupid")
    set memberOf = ##class(EasyCSP.Core.Array).%New()
    do memberOf.split(myGroups, $mvvm)

The top line gets the value from the USERS GROUP_ID attribute (called groupid in the file class). The next line makes a new EasyCSP.Core.Array. (I'll attach the code at the bottom.) The third line splits the multivalued attribute into an array along @VM lines. The same thing in MVBasic would be:

memberOf = "EasyCSP.Core.Array"->%New()

READ myself FROM USERS, someId THEN

    myGroups = myself<groupIdAttribute> ; * ignore the fact that the original code is getting the value from a data model

   memberOf->split(myGroups, @vm)

END

Later in the code where I want to see if any of the user's groups are in the access rule's list of groups that are allowed to run a particular screen (web page), I do this:

        if 'groupGood {
            set allowedGroups = ##class(EasyCSP.Core.Array).%New()
            do allowedGroups.splitList(rule.groups) // split the list into an array
            set matches = allowedGroups.intersect(memberOf)
            set:matches.Count()>0 groupGood = 1
            quit
        }

-or-

IF NOT(groupGood) THEN

   allowedGroups = ##class(EasyCSP.Core.Array)->%New()

   allowedGroups->splitList(rule.groups) ; * my rule object holds allowed groups as a Cache' %List

   matches = allowedGroups->intersect(memberOf)

   if matches.Count()>0 then groupGood = 1

   EXIT ; * the loop searching through all the rules to find one that works

END

Two completely different structures can meet in the middle for an easy method to compare the contents of the two. This also works great for two long dynamic arrays where you want to find values in common or different between one array and another.

This is part of my EasyCSP site-building framework but it has become handy enough to share as a standalone.

Enjoy,

Bill

class EasyCSP.Core.Array extends %RegisteredObject

Extends %ListOfDataTypes with some Python/Javascript/PHP -like features

Copyright (c) 2011 James W "Bill" Westley-Farrell

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Apache License 2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Inventory

Parameters	Properties	Methods	SystemMethods	Queries	Indices	ForeignKeys	Triggers
		21

Summary

Methods
%AddToSaveSet	%ClassIsLatestVersion	%ClassName	%ConstructClone
%DispatchClassMethod	%DispatchGetModified	%DispatchGetProperty	%DispatchMethod
%DispatchSetModified	%DispatchSetMultidimProperty	%DispatchSetProperty	%Extends
%FromJSONValue	%FromObject	%GetParameter	%IsA
%IsModified	%New	%NormalizeObject	%ObjectModified
%OriginalNamespace	%PackageName	%RemoveFromSaveSet	%SerializeObject
%SetModified	%ToAbstractObject	%ToDynamicArray	%ToDynamicObject
%ToJSONValue	%ValidateObject	Count	Next
append	clear	diff	dumpAsStream
find	flip	get	intersect
join	length	pop	remove
set	shift	split	splitList
toDynamicArray	toJSON	unshift

Methods

• method Count() as %Integer

Traditional-Cache alias for .length(). This returns the number of elements in the array.

• method Next(key As %Integer = 0) as %Integer

Finds and returns the index value of the element at the location following key in the list. If key is a null string (""), then Next returns the position of the first element in the list (1).

• method append(value As %String = "")

`Add a value to the end of the array

• method clear()

Clear out the array structure

• method diff(compareArray As EasyCSP.Core.Array) as EasyCSP.Core.Array [ Language = mvbasic ]

Returns an array containing all the values that do not exist in both this array and the array passed in. Neither array is changed.

• method dumpAsStream() as %GlobalCharacterStream

Dump the contents of the array in a two-column (key and data) formatted HTML table. Returns a stream that can be output to a web page.

• method find(target As %String = "") as %Integer

Given a string somewhere in the array, return the node where it was found or 0 if not.

• method flip() as EasyCSP.Core.Array

Reverse the order of the values in the array. Returns a new array leaving the original unchanged.

• method get(position) as %String

Retrieve the value from a specific node in the array.

• method intersect(compareArray As EasyCSP.Core.Array) as EasyCSP.Core.Array [ Language = mvbasic ]

Returns an array containing all the values that exist in both this array and the array passed in. Neither array is changed.

• method join(delimiter As %String = ",") as %String [ Language = mvbasic ]

Returns the elements of the array in a delimited string. If a delimiter isn't supplied a comma is used. Any string of any length can be passed in delimiter.

• method length() as %Integer

JQuery/Python-like alias for Count()

• method pop() as %String

Pop a value off the end of the array and shorten the array by one.

• method remove(key As %Integer = 0)

Removes an element and its key completely.

• method set(position As %Integer, value As %String)

Set the value at a specific node in the array. If the position argument is -1, this works the same as .append(). If the position doesn't exist it will be created. If the position has a value, that value will be overwritten.

• method shift() as %String

Shifts a value off the beginning of the array and shortens the array by one. Returns the shifted value. This works the same as PHP array_shift().

• method split(source As %String = "", delimiter As %String = ",") [ Language = mvbasic ]

Split a delimited string into an array. If delimiter is not supplied a comma is assumed. This does not strip spaces around elements in the delimited string. This is useful for splitting comma-delimited strings or dynamic arrays into an array structure. For instance, to compare the contents of a dynamic array with a comma-separated list, split both lists into an EasyCSP.Core.Array then use .intersect() or .diff() to return matches (or non- matches in the case of .diff()).

• method splitList(source As %List = "") [ Language = mvbasic ]

Split a Cache list structure into an array

• method toDynamicArray(quoteStrings As %Boolean = 0) as %String [ Language = mvbasic ]

Returns a string representation of the elements in the array delimited by attribute marks.

• method toJSON() as %String [ Language = mvbasic ]

Returns the elements of the array as JSON.

• method unshift(element As %String) as %Status

Adds a value to the beginning of the array and shifts the original elements up by one.

---

And here's the code:

/// Extends %ListOfDataTypes with some Python/Javascript/PHP -like features
Class EasyCSP.Core.Array Extends %RegisteredObject
{

Property Data As list Of %CacheString;

Property Size As %Integer [ InitialExpression = 0, Private ];

/// `Add a value to the end of the array
Method append(value As %String = "")
{
    do ..set(-1, value)
}

/// Clear out the array structure
Method clear()
{
    kill i%Data
    set ..Size = 0
}

/// Traditional-Cache alias for .length(). This returns the number
/// of elements in the array.
Method Count() As %Integer
{
    quit i%Size
}

/// Returns an array containing all the values that do not exist in
/// both this array and the array passed in. Neither array is
/// changed.
Method diff(compareArray As EasyCSP.Core.Array) As EasyCSP.Core.Array [ Language = mvbasic ]
{
    thisDyn = @ME->toDynamicArray()
    thisDynLength = DCount(thisDyn, @fm)
    
    compareDyn = compareArray->toDynamicArray()
    compareDynLength = DCount(compareDyn, @fm)
    
    if thisDynLength <= compareDynLength then
        a = compareDyn
        b = thisDyn
    l = compareDynLength
    end else
        a = thisDyn
        b = compareDyn
        l = thisDynLength
    end
    
    * Use the shorter array to traverse.
    newArray = "EasyCSP.Core.Array"->%New()
    for idx = 1 to l
        find a<idx> in b setting AMC else
            newArray->append(a<idx>)
        end
    next
    
    return newArray
}

/// Dump the contents of the array in a two-column (key and data)
/// formatted HTML table. Returns a stream that can be output to
/// a web page.
Method dumpAsStream() As %GlobalCharacterStream
{
    set stream = ##class(%GlobalCharacterStream).%New()
    
    do stream.WriteLine("<table>")
    
    set idx = ..Data.Next("")
    while idx '= "" {
        set val = ..get(idx)
        do stream.WriteLine("<tr>")
        do stream.WriteLine("<td>"_idx_"</td>")
        do stream.Write("<td>")
        if $isobject(val) {
            do stream.Write("[object]")
        } else {
            do stream.Write(val)
        }
        do stream.Write("</td>")
        do stream.WriteLine("</tr>")
        set idx = ..Data.Next(idx)
    }
    
    do stream.WriteLine("</table>")
    
    quit stream
}

/// Given a string somewhere in the array, return
/// the node where it was found or 0 if not.
Method find(target As %String = "") As %Integer
{
    set o = $order(i%Data(""))
    while o '= "" {
        if $get(i%Data(o)) = target return o
    }
    quit 0
}

/// Reverse the order of the values in the array. Returns
/// a new array leaving the original unchanged.
Method flip() As EasyCSP.Core.Array
{
    set newArray = ##class(EasyCSP.Core.Array).%New()
    for i = i%Size:-1:1 {
        do newArray.append(i%Data(i))
    }
    quit newArray
}

/// Retrieve the value from a specific node in the array.
Method get(position) As %String
{
    quit:+position=0 "" // if not numeric
    quit:'$data(i%Data(position)) "" // if not defined
    quit $get(i%Data(position))
}

/// Returns an array containing all the values that exist in
/// both this array and the array passed in. Neither array is
/// changed.
Method intersect(compareArray As EasyCSP.Core.Array) As EasyCSP.Core.Array [ Language = mvbasic ]
{
    thisDyn = @ME->toDynamicArray()
    thisDynLength = DCount(thisDyn, @fm)
    
    compareDyn = compareArray->toDynamicArray()
    compareDynLength = DCount(compareDyn, @fm)
    
    if thisDynLength >= compareDynLength then
        a = compareDyn
        b = thisDyn
    l = compareDynLength
    end else
        a = thisDyn
        b = compareDyn
        l = thisDynLength
    end
    
    * Use the shorter array to traverse.
    newArray = "EasyCSP.Core.Array"->%New()
    for idx = 1 to l
        find a<idx> in b setting AMC then
            newArray->append(a<idx>)
        end
    next
    
    return newArray
}

/// Returns the elements of the array in a delimited string. If
/// a delimiter isn't supplied a comma is used. Any string of
/// any length can be passed in <var>delimiter</var>.
Method join(delimiter As %String = ",") As %String [ Language = mvbasic ]
{
    @ME->toDynamicArray
    
    rtnDyn = ereplace(rtnDyn, @am, delimiter)
    
    return rtnDyn
}

/// JQuery/Python-like alias for Count()
Method length() As %Integer
{
    quit i%Size
}

/// Finds and returns the index value of the element at the location following <var>key</var> in the list.
/// If key is a null string (""), then <b>Next</b> returns the position of the first element in the list (1).
Method Next(key As %Integer = 0) As %Integer
{
    quit $order(i%Data(key))
}

/// Pop a value off the end of the array and shorten the
/// array by one.
Method pop() As %String
{
    quit:i%Size<1 ""
    set value = i%Data(i%Size)
    kill i%Data(i%Size)
    set i%Size = i%Size - 1
    quit value
}

/// Removes an element and its key completely.
Method remove(key As %Integer = 0)
{
    quit:+$get(key)=0
    kill i%Data(key)
}

/// Set the value at a specific node in the array. If the position argument is
/// -1, this works the same as .append(). If the position doesn't exist it will
/// be created. If the position has a value, that value will be overwritten.
Method set(
position As %Integer,
value As %String)
{
    set new = 0
    
    // append immediately to the end
    set:$get(position)="" position = -1
    //if (position < 1) ! (position > i%Size) {
    if (position < 1) {
        set position = i%Size + 1
        set new = 1
    }
       
    set i%Data(position) = value
    set:new i%Size = i%Size + 1
}

/// Shifts a value off the beginning of the array and shortens
/// the array by one. Returns the shifted value. This works the
/// same as PHP array_shift().
Method shift() As %String
{
    set o = $order(i%Data(""))
    quit:o="" ""
    set value = i%Data(o)
    kill i%Data(o)
    set i%Size = i%Size - 1
    quit value
}

/// Split a delimited string into an array. If <variable>delimiter</variable> is
/// not supplied a comma is assumed. This does not strip spaces around elements
/// in the delimited string. This is useful for splitting comma-delimited strings
/// or dynamic arrays into an array structure. For instance, to compare the contents
/// of a dynamic array with a comma-separated list, split both lists into an
/// EasyCSP.Core.Array then use .intersect() or .diff() to return matches (or non-
/// matches in the case of .diff()).
Method split(
source As %String = "",
delimiter As %String = ",") [ Language = mvbasic ]
{
    if $get(source) = "" then return
    @ME->clear()
    if $get(delimiter) = "" then delimiter = ","
    convert delimiter to @fm in source
    d = dcount(source, @fm)
    for idx = 1 to d
        @ME->append(source<idx>)
    next
}

/// Split a Cache list structure into an array
Method splitList(source As %List = "") [ Language = mvbasic ]
{
    if not($listValid(source)) = "" then return
    source = $listToString(source, @fm)
    @ME->clear()
    d = dcount(source, @fm)
    for idx = 1 to d
        @ME->append(source<idx>)
    next
}

/// Returns a string representation of the elements in the array
/// delimited by attribute marks.
Method toDynamicArray(quoteStrings As %Boolean = 0) As %String [ Language = mvbasic ]
{
    rtnDyn = ""
    
    key = @ME->Next("")
    
    loop while key <> "" do
    
        value = @ME->get(key)
        
        if quoteStrings then
        
            if (oconv(value, "MCN") <> value) then
                value = squote(value)
            end
            
            if value = "" then value = '""'
        end
        
        rtnDyn<-1> = value
        key = @ME->Next(key)
        
    repeat
    
    return rtnDyn
}

/// Returns the elements of the array as JSON.
Method toJSON() As %String [ Language = mvbasic ]
{
    nvPairs = @ME->toDynamicArray(1)
    convert @vm:@fm to ":," in nvPairs
    return "[" : nvPairs : "]"
}

/// Adds a value to the beginning of the array and shifts the
/// original elements up by one.
Method unshift(element As %String) As %Status
{
    For i=i%Size:-1:1 Set i%Data(i+1)=i%Data(i)
    Set i%Data(1)=element,i%Size=i%Size+1
    Quit $$$OK
}

Storage Custom
{
<Type>%Library.CompleteCustomStorage</Type>
}

}

[Michael Cohen]

Bill,

Really interesting, and thanks for posting.

But I am confused by one thing:  this seems to nicely handle JSON name/value pairs, but is name ( 'key') integer or string?

there is:

method Next(key As %Integer = 0) as %Integer

but I also see:

Method get(key As %String) As %String

Or do I just need to read more carefully?

[James Westley Farrell]

Generally in Cache', a %ListOfSomething has alphanumeric subscripts (at least internally) and an %ArrayOfSomething has numeric subscripts, even if they're only implied.

In JSON, an array is a collection of items. Implicitly those have numeric subscripts as well; 

ary =  [ "a", 1, "x", 5, ...];

x = ary[1]; // would be 1

In JSON (and Python dictionaries) a collection is a set of name-value pairs.

dict = { "sub1": "a", "sub2": 1, "sub3": "x", ...};

x = dict["sub1"] ; would be "a"

Python gives you a nice .iter() function that allows you to iterate a collection. Cache' has .Next() which gives you sort-of the same thing.

Since I'm working on a site-building add-on for Cache', I'm having to deal with the differences between native Cache' collections and what JS and JQuery want them to look like. For me, it was easier to extend Cache's array and list a little bit to make them match neatly.

So, a core Array looks and feels like a Javascript/Python array (.toJSON would give you ["thing", "thing", "thing"...] and a core HashArray looks and feels like a JS/Python object/dictionary, { "firstSub": "thing", "secondSub": "anotherThing"...} and so on.

Hope that makes it a bit clearer and gives you some ideas where either might be useful.

To be truthful, at the current release, either .toJSON() function only goes a level deep. I plan to allow them to recurse, at least in HashArray when they detect the value at a node is another list or array object. I haven't needed it yet but it would only take a few minutes to code when I do.

I'm also greedy <lol>. While I was on about it, it seemed that it was easy enough to give myself some PHP-like array functions, equivalent to array_intersect(), array_union, and array_diff(). It turned out those were really handy for comparing contents of dynamic arrays. A construct like: "Here are two arrays or lists: give me all the elements that do (not) exist in both." When you have two disparate kinds of list (say, for instance, a dynamic array and a %List), it's a no-brainer to load two arrays, one by a list and one with a dynamic array then say:

SomeList = $LISTBUILD("a", "b", "x", "q")

ArrayFromList = "EasyCSP.Core.Array"->%New()

ArrayFromList->splitList(SomeList) ; * puts the contents of the list ordered into the array

SomeDyn = "a" : @vm : "b" : @vm : "r" : @vm : "FortyTwo"

ArrayFromDyn = "EasyCSP.Core.Array"->%New()

ArrayFromDyn->split(SomeDyn, @vm)

NewArray = ArrayFromList.diff(ArrayFromDyn) ; * give me the differences

would give you a new array containing ["q", "r", "x", "FortyTwo"]

--Bill

"He is your friend, your partner, your defender, your dog. You are his life, his love, his leader. He will be yours, faithful and true, to the last beat of his heart. You owe it to him to be worthy of such devotion" -- Unknown

--
You received this message because you are subscribed to the Google Groups "InterSystems: MV Community" group.
---
You received this message because you are subscribed to the Google Groups "InterSystems: MV Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to InterSystems-...@googlegroups.com.
To post to this group, send email to InterSy...@googlegroups.com.
Visit this group at https://groups.google.com/group/InterSystems-MV.
For more options, visit https://groups.google.com/d/optout.