JSON functionality in mumps

[Kevin Toppenberg]

I needed ability to convert mumps array format into json and back again.

I found the XLFJSON library with ENCODE and DECODE functions.  But they don't seem to work properly, as shown below.

yottadb>set ZZ(0)=0

yottadb>SET ZZ(1)=1

yottadb>SET ZZ(2)=2

yottadb>SET ZZ(1,2,3)="MISSING"

yottadb>ZWR ZZ
ZZ(0)=0
ZZ(1)=1
ZZ(1,2,3)="MISSING"
ZZ(2)=2

yottadb>DO ENCODE^XLFJSON("ZZ","OUT")

yottadb>ZWR OUT
OUT(1)="{""0"":0,""1"":1,""2"":2}"

yottadb>DO DECODE^XLFJSON("OUT","NEWZZ")

yottadb>ZWR NEWZZ
NEWZZ("""0")=0
NEWZZ("""1")=1
NEWZZ("""2")=2

Notice that the "MISSING" node I initially added into ZZ was not included in the output json.  Also notice that when recreating the array, the nodes have an extra unwanted quote character.

So, in my tradition of reinventing the wheel, I have spent the past 3 days working on replacement code.  I thought about trying to fix the XLFJSON codebase, but sometimes its easier to start over than trying to fix someone else's code. 

My version

https://github.com/kdtop/TMGLIB/blob/master/TMGJSON.m

Here is my code working using the same ZZ variable from above. 

yottadb>SET STR=$$ARR2JSON^TMGJSON("ZZ")

yottadb>W STR
[0,{"%%node_value%%":1,"2":{"3":"MISSING"}},2]
yottadb>

yottadb>KILL NEWZZ DO JSON2ARR^TMGJSON(STR,"NEWZZ")

yottadb>ZWR NEWZZ
NEWZZ(0)=0
NEWZZ(1)=1
NEWZZ(1,2,3)="MISSING"
NEWZZ(2)=2

Kevin

[Ed de Moel]

There is a difference between M[UMPS] and the languages from which JSON descends:
In M[UMPS] any data element can have both a value and descendants.
In JSON, a data element can EITHER have a value OR have descendants... not both.

Hope this helps,
Ed

[OldMster]

I've attempted various times to make a generic array <--> json convertor, but have generally not been successful.  Like you, I had to come up with a indicator of the node value when it had descendants, and I like your approach.  The issue I have consistently run up against is intelligently switching between an object and an array in json.  Too often what I want is not obvious from the context of the data, it requires knowledge of the intended use of the data to decide which to use.

[Nancy Anthracite]

Why am I not surpirsed! Thanks!

--

Nancy Anthracite

[Coty Embry]

Ahh yes the famous sparse array to json conversion problem. 

I handled this problem by writing various recursion routines based on each different sparse array structure. 

Never wrote a one size fits all routine. Just wrote a different function for each sparse array format to spit out the json

Goodluck on your adventure. If you need any of my stuff let me know!

On May 29, 2025, at 3:58 PM, Nancy Anthracite <nanth...@earthlink.net> wrote:



--
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+u...@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "Hardhats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hardhats/5028676.bCDJkKcVGE%40owl.

[Ed de Moel]

Yes. Unfortunately, the correct approach is also the most inefficient...

The correct approach is to represent each node in a M[UMPS] structure as an object,
let's call it "node", and each node is a {"Value": whatever or null, "Descendants": whatever object that is either null or a node}.

Now, of course, this would make for an extremely verbose JSON stream, especially since, in most cases, either the Value or the Descendants are a "null".

So, I agree with one of the previous respondents: the approach you should choose depends what you know about  the actual structure you're dealing with.

Ed

To view this discussion visit https://groups.google.com/d/msgid/hardhats/486D2601-B896-486B-B936-B33C1421D60F%40gmail.com.

--
Ed de Moel
--
**************************************************************
This electronic mail transmission contains confidential and/or
privileged information intended only for the person(s) named.
Any use, distribution, copying or disclosure by another person
is strictly prohibited.
**************************************************************

Virus-free.www.avg.com

[Ar “art”]

take a look at        VA FileMan DDE Utility.      April 2023

1 Introduction

The ENTITY (#1.5) file can map Veterans Health Information Systems and Technology Architecture

(VistA) files and fields to any data model, including common standards such as Health Level Seven

(HL7) Fast Healthcare Interoperability Resources (FHIR) or InterSystems’ Summary Document

Architecture (SDA). The VA FileMan DDE utility uses the ENTITY (#1.5) file as a template for

producing eXtensible Markup Language (XML) or JavaScript Object Notation (JSON) output from

VistA data.

1.1 What is an Entity?

An Entity is simply a collection of data elements, similar to a data class in an object-oriented system. If

a specific data model is being followed, that model determines the tag names and allowable values of

each element. The Entity assigns each tag to a file and/or field in VistA, transforming the data as needed

to meet any requirements or constraints of the model.

Usually, an Entity returns a single record from a VistA file, accepting a record Internal Entry Number

(IEN) and returning the defined fields as the data elements. An Entity can accept any string as its input,

however; for example, an Entity could be created to take a VistA name string and return its components

as separate elements within a group.

Like classes, entities can be nested. Commonly used data elements, such as locations, for example, can

have their own Entity that accepts a pointer to the HOSPITAL LOCATION (#44) file and returns a

group of attributes (code and description, specialty, phone#, etc.) from that file, which can then be

embedded in other entities.

1.2 VA FileMan DDE Utility

DDE is the VA FileMan utility that uses an Entity to build XML or JSON output. The ENTITY (#1.5)

file is stored in the ^DDE global. Entry points $$GET1 and GET in routine DDE are supported calls

that can be used to return VistA data, one or more records at a time, according to the specification

defined by the Entity.

The exercises in this tutorial use the Data Mapping [DDE ENTITY MAPPING] options on the Other

Options [DIOTHER] menu in VA FileMan.

From VA DOC. This could be helpful 

art

[Kevin Toppenberg]

Thank you all for the replies.  Several replies indicated an understanding that I am still working on this.  Actually, I have finished it, and it is ready to go.  Does that mean bugs won't be found?  No, but its to the point that I am going to start using it.  I encourage anyone else to try it out too. 

Thanks again

Kevin

[rtweed]

You'll find that JSON mapping is something I tackled long, long ago - see my various repos at https://github.com/robtweed. In particular if you're wanting an M-centric solution, see the logic that's built into my mgweb-server project: https://github.com/robtweed/mgweb-server

if your focus is JavaScript (which I know it isn't, but mine is), then take a look at how my glsdb project handles and abstracts persistence and recovery: https://github.com/robtweed/glsdb

The problem is whether your starting point is existing M data that you want to map in and out of JSON, or if your starting point is JSON data that you want to persist.  

My focus has always been on the latter.  

If your focus is the former, then you quickly hit the issue of how to represent arrays in an unambiguous way, and how to deal with storage of data at intermediate nodes rather than always on leaf nodes.  Basically there's no "one" solution for these issues: whatever you do will ultimately be a bit of a kludge.

However if your starting point is persistence of JSON in M globals and their recovery into JSON again, you could use my tried and tested solutions rather than reinventing the wheel.