FileMan Only with Print Implementation

[Benjamin Irwin]

While attempting to watch the Boeing Starliner liftoff, and finally seeing the liftoff, and waiting for the docking, which took a lot of devoted watching time, I created a stand alone FileMan from my implementation of Linux/GT.m/VistA.

It is available at https://edu2.opensourcevista.net/vista/index.php.  In the download list at the bottom of the page.

I haven't done extensive testing, but it mostly works from what I have tested.

There is a problem when printing a dictionary definition to a linux file ( DATA DICTIONARY UTILITIES, LIST FILE ATTRIBUTES).

It seems to forget its environment variables.

Feel free to download and try at your own risk.  Let me know if you can help me fix the print issues.

Thanks,

Ben

[David Whitten]

The big issue is getting I/O working, ie ^%ZIS. 

It is a pain, but can be done.

What did you use for those routines ?

Dave Whitten

713-870-3834

--
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+unsubscribe@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "Hardhats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/5e6afc5a-5d5a-4827-b871-63d502a07a0dn%40googlegroups.com.

[Rob Kellock]

Hi,

I've been lurking on this Group for ages, but after stumbling over this YouTube video https://www.youtube.com/watch?v=Pyu70gFxxtQ from Matthew Wilson (thanks, Matthew. It's perfect for newbies), I finally decided that by following along I might be able to overcome the steep learning curve that is Mumps. Job done! So now I have Medsphere version 1062 FileMan ( https://hardhats.org/fileman/MSC.html ) running stand alone on Reference Standard M. This IMO is a good way for newbies to get their toes wet with M.

I MIGHT have found a bug, but being so new, I haven't tried yet to locate it in the code or to reproduce it on another M database e.g. Yotta, so I'm not sure whether it is in FileMan or Reference Standard M (I inadvertently created an endless loop in RSM too which consumed 100% of my CPU, but that's another story).

Under the menu item Data Dictionary Utilities/List File Attributes/Indexes Only 

the Report Heading is duplicating:

Select LISTING FORMAT: STANDARD// iNDEXES ONLY  
What type of cross-reference (Traditional or New)? Both// b  BOTH
Which field: ALL//
INDEX AND CROSS-REFERENCE LIST -- FILE #999001INDEX AND CROSS-REFERENCE LIST -- FILE #9990016/15/24     PAGE 1
-------------------------------------------------------------------------------

File #999001

Is this related to the environment variables too, Ben?

Thanks,

Rob.

[Benjamin Irwin]

Rob,

From a FOIA Ubuntu/GT.m/VistA system, I am getting the following just like you mentioned.

Select OPTION: 8  DATA DICTIONARY UTILITIES
Select DATA DICTIONARY UTILITY OPTION: 1  LIST FILE ATTRIBUTES
 START WITH What File: DEVICE// 9.2  HELP FRAME  (766 entries)
      GO TO What File: HELP FRAME//       (766 entries)
      Select SUB-FILE:
Select LISTING FORMAT: STANDARD// INDEXES ONLY  
What type of cross-reference (Traditional or New)? Both// BOTH
Which field: ALL//
DEVICE: HOME//   TELNET
INDEX AND CROSS-REFERENCE LIST -- FILE #9.2INDEX AND CROSS-REFERENCE LIST -- FIL
E #9.2                                                      6/14/24     PAGE 1
-------------------------------------------------------------------------------

From a FOIA Windows 10/Cache/VistA system, I am getting the following just like you mentioned.

Select OPTION: 8  DATA DICTIONARY UTILITIES
Select DATA DICTIONARY UTILITY OPTION: 1  LIST FILE ATTRIBUTES
 START WITH What File: REMOTE PROCEDURE// 9.2  HELP FRAME
                                          (766 entries)
      GO TO What File: HELP FRAME//       (766 entries)
      Select SUB-FILE:
Select LISTING FORMAT: STANDARD// INDEXES ONLY
What type of cross-reference (Traditional or New)? Both// BOTH
Which field: ALL//
DEVICE: HOME//   Console (Cache' on Windows)    Right Margin: 80//
INDEX AND CROSS-REFERENCE LIST -- FILE #9.2INDEX AND CROSS-REFERENCE LIST -- FIL
E #9.2                                                      6/14/24     PAGE 1
-------------------------------------------------------------------------------

This appears to work consistently across different environments.  That would lead me to believe this is written in the FileMan MUMPS code.

Ben

[David Wicksell]

Hi Rob,

Yep, Matthew's videos are great. So it looks like there was not a bug in RSM after all. I'm the

developer and maintainer of RSM, and you mentioned you caused an endless loop in RSM.

It's trivially easy to create an endless loop that consumes 100% of a CPU core in any M

implementation. However, if you think that it was caused by a bug in RSM, and you know

how to replicate it, please let me know so I can fix it right away. I'm glad you're enjoying

RSM. Thank you.

David Wicksell

Fourth Watch Software LC

[Rob Kellock]

Thanks David,

It's trivially easy to create an endless loop that consumes 100% of a CPU core in any M

implementation.

I was going to say it does seem to be easy to do, but now I can't replicate the problem. Previously, I must have set a variable without realising it and then the following command tried to use it. This is what happens when you cut and paste code snippets without understanding what they do :-\ .

I'm thinking of looking through the FileMan code in a brave attempt to locate the bug. Hopefully its not full of gotos like the 1000 line functions I had to debug 25 years ago in a Pick database!

How do people generally maintain Mumps code?  My inclination would be to export the global ( ^DI ) to disk from RSM's REPL , explore it using a text editor and then use FileMan to alter the appropriate routine. Is there a better way?

Cheers,

Rob.

[George Timson]

When I try to LIST FILE ATTRIBUTES in the way shown here by Rob Kellock, I get the expected output:

INDEX AND CROSS-REFERENCE LIST -- FILE #9.2INDEX AND CROSS-REFERENCE LIST -- FILE #9.2                                                     6/15/24     PAGE 1
-------------------------------------------------------------------------------

File #9.2

  Traditional Cross-References:

  AC    REGULAR
            Field:  AUTHOR  (9.2,6)
                    1)= S ^DIC(9.2,"AC",$E(X,1,30),DA)=""
                    2)= K ^DIC(9.2,"AC",$E(X,1,30),DA)

  B    REGULAR
            Field:  NAME  (9.2,.01)
                    1)= S ^DIC(9.2,"B",$E(X,1,30),DA)=""
                    2)= K ^DIC(9.2,"B",$E(X,1,30),DA)

  C    KWIC
            Field:  HEADER  (9.2,1)
                    1)= F %=1:1 Q:$P(X," ",%,99)=""  S I=$P(X," ",%) I $L(I)>2,
                    $L(I)<31,^DD("KWIC")'[I S ^DIC(9.2,"C",I,DA)=""
                    2)= F %=1:1 Q:$P(X," ",%,99)=""  S I=$P(X," ",%) I $L(I)>2,
                    $L(I)<31 K ^DIC(9.2,"C",I,DA)

  D    REGULAR
            Field:  DATE ENTERED  (9.2,4)
                    1)= S ^DIC(9.2,"D",$E(X,1,30),DA)=""
                    2)= K ^DIC(9.2,"D",$E(X,1,30),DA)

...

and so on. I start with

>D P^DI

What are you starting with?

(I admit that the double-display of the words 'INDEX AND CROSS-REFERENCE LIST -- FILE #9.2' is a surprise to me, and not intended.  An ugly cosmetic bug that must be several decades old.)

BTW, the global ^DI is used by FIleMan to store certain Files, NOT to store routine code, as Kellock seems to think.

[Rob Kellock]

Hi George,

Yes, I'm using D P^DI

(I admit that the double-display of the words 'INDEX AND CROSS-REFERENCE LIST -- FILE #9.2' is a surprise to me, and not intended.  An ugly cosmetic bug that must be several decades old.)

 That's the bug I'm referring to.  You can always guarantee that a new user will unerringly ferret out something unexpected :-)

BTW, the global ^DI is used by FIleMan to store certain Files, NOT to store routine code, as Kellock seems to think.

Is the source code is stored in the database somewhere? As previously said I have Pick experience from years ago where code was stored in the Pick database along with data (I vaguely recall some kind of pcode compiling process occurred where a compiled record had the same name as a source code record with a different suffix to distinguish them e.g. code.str and code.cpl and we stripped out the code.str records when delivering an update to a client). I assumed that M had a similar idea. I know it's not looking back into MSCFILEMAN_1062.RSA, because I've just renamed that to something else and FileMan stills runs. Is the source code for FileMan being compiled and placed in $ROUTINES? Does the VA maintain all its source code in text files outside of the database? Apologies for the newbie questions.

[David Wicksell]

Hello Rob,

 So, M language implementations do have databases, of course. They are there
primarily as a backing implementation for globals. However, they are not
specified, nor even mentioned in the M language standard, which though 25 years
old, is still in effect as an ISO standard. The standard leaves a lot of things
up to the implementers. It specifies the syntax and semantics of globals, but
how they are implemented is up to the implementers. Another thing that is not
part of the standard, is routine management.

 Through the years, there have been two primary approaches to routine storage.
As you surmised, some implementations have stored routines in globals, or
structured system variables (SSVN), which would then put them in a database (or
data storage engine, which is probably closer to what an M database should be
thought of). While some have stored the routines directly in the execution
environment. Back in the 70s and 80s, an M system ran directly on the bare
metal, and wasn't just the language environment, but also managed all of the
computer resources directly, functioning as the operating system. I'm sure you
know some or all of this, but I mention it just as a quick background to
answering your questions. So in the implementations from that era, routines
were stored in globals, and SSVNs didn't exist yet.

 But over time, as general purpose operating systems became more popular, after
the proliferation of UNIX throughout the 70s, and then MS and Apple OSes
becoming popular in the 80s, M implementations started being implemented on top
of another operating system, and it became more interesting to consider storing
routines directly in the file system of those operating systems. So with that
quick, and overly simplistic, history out of the way, the landscape today, has
roughly seven viable M implementations. By that I mean ones that are currently
maintained and have some level of new development. They are split roughly down
the middle on the two different approaches.

 So to answer your question, some implementations store M source in a global or
SSVN, and therefore in a database, while others store M source as files on
disk. As far as the question about stripping out source before shipping an
application, some implementations allow that, and some don't. Here, I need to
mention that the implementation techniques differ between the implementations.
Of the ones that are more or less current, there is a direct interpreter, a
p-code compiler with an interpreter (which now would usually be called a
bytecode compiler with a virtual machine of differing types), and there are
compiled implementations with JIT compilers, that store routines as native
machine code. I believe most of the implementations do allow you to strip out
the source code, but a direct interpreter cannot do that, since it only stores
routines as source. Also, of note, there is a standard language feature that
makes it very difficult to strip out source, the $TEXT function, which returns
lines of source code.

 My implementation, Reference Standard M (RSM), is a bytecode compiler with a
virtual machine interpreter, which stores routines in the ^$ROUTINE SSVN, in the
database. It stores the bytecode in the 0 node, and the source in nodes 1
through N. You cannot write to ^$ROUTINE directly, rather you add or modify
source by writing to nodes (1 through N, as a monotonically increasing integer)
in a local variable array, and then you MERGE them in to ^$ROUTINE(<routine
name>), and the MERGE will store the source, and compile and store the bytecode.
There is a convenience utility, called ^%ED, which will open up an editor of
your choice, stored in the $EDITOR environment variable, or defaulting to vi,
with a copy of the source for a routine, and when you make changes and write
them and quit out of the editor, it will then MERGE your changes back in to
^$ROUTINE, and update, and recompile your code. You cannot remove the source,
because that would break $TEXT, as RSM does not store source, in whole or in
part, in the bytecode.

 When you execute a routine, that routine is placed in to routine buffers in
shared memory. They are shared among all RSM jobs, with automated procedures for
managing different currently executing routine versions, so that nothing is
clobbered. I hope that answers your general and specific questions. If you want
to go deeper with RSM, shoot me an email at d...@linux.com. Thank you.

David Wicksell
Fourth Watch Software LC

[Rob Kellock]

Thanks David,

I appreciate this very detailed explanation of the different flavours of M when it comes to managing source code. There's a lot to unpack here. When I first installed FileMan, I thought I was about to start using an industrial strength Microsoft Access, created long before Microsoft existed. Now I see that isn't what FileMan is about.

Thanks again for your responses.

Rob.

[Rob Kellock]

Hi George,

At line 31385 in MSCFILEMAN_1062.RSA there is this line inside a function (maybe in M you refer to functions by another name) called IXHEAD

 W:$D(DIFF)&($Y) @IOF S DIFF=1 W "INDEX AND CROSS-REFERENCE LIST -- FILE #"_DIB,?(IOM-20),$$OUT^DIALOGU(DT,"FMTE",2)_"     "_$$EZBLD^DIALOG(7095,DC) ;**CCO/NI DATE FORMAT, 'PAGE'

I changed it to:

 W:$D(DIFF)&($Y) @IOF S DIFF=1 W ?(IOM-19),$$OUT^DIALOGU(DT,"FMTE",2)_"     "_$$EZBLD^DIALOG(7095,DC) ;**CCO/NI DATE FORMAT, 'PAGE'

That seemed to fix the bug.

Cheers,

Rob

[Benjamin Irwin]

That code seems to be in line 124 of routine DID (IXHEAD+3)

DID ;SFISC/XAK-LIST DD'S ;2015-01-02  12:25 PM
 ;;22.2;VA FileMan;;Jan 05, 2016;Build 42
 ;;Per VA Directive 6402, this routine should not be modified.
 ;;Submitted to OSEHRA 5 January 2015 by the VISTA Expertise Network.
 ;;Based on Medsphere Systems Corporation's MSC FileMan 1051.
 ;;Licensed under the terms of the Apache License, Version 2.0.
.

.

.

IXHEAD S DC=DC+1 I IOST?1"C".E W $C(7) R M:DTIME S:'$T M=U Q:M=U

IXHEAD1 W:$D(DIFF)&($Y) @IOF S DIFF=1
 W $S("B"[$G(DIDTYP):"INDEX AND CROSS-REFERENCE",DIDTYP="T":"TRADITIONAL CROSS-REFERENCE",1:"NEW-STYLE INDEX")
 W " LIST -- FILE #"_DIB_$S($G(DIDFLD):", FIELD #"_DIDFLD,1:"")

 W:$D(DIFF)&($Y) @IOF S DIFF=1 W "INDEX AND CROSS-REFERENCE LIST -- FILE #"_DIB,?(IOM-20),$$OUT^DIALOGU(DT,"FMTE",2)_"     "_$$EZBLD^DIALOG(7095,DC) ;**CCO/NI DATE FORMAT, 'PAGE'

 S M="",$P(M,"-",IOM)="" W !,M
 Q

[George Timson]

Yes, that subroutine, IXHEAD^DID is what needs to be fixed.  Good.  But Rob Kellock's fix is incomplete; when there is more than one page of 'INDEXES ONLY' output, the header on the 2nd page is screwed up.  Here's what I think works:

IXHEAD1 W !,@IOF

 W $S("B"[$G(DIDTYP):"INDEX AND CROSS-REFERENCE",DIDTYP="T":"TRADITIONAL CROSS-REFERENCE",1:"NEW-STYLE INDEX")
 W " LIST -- FILE #"_DIB_$S($G(DIDFLD):", FIELD #"_DIDFLD,1:"")

 W ?(IOM-20),$$OUT^DIALOGU(DT,"FMTE",2)_" "_$$EZBLD^DIALOG(7095,DC) ;**CCO/NI DATE FORMAT, 'PAGE'
  ...  and so on

[Benjamin Irwin]

I have made George's suggested changes to lines 121 and 124 that can be seen in the DDI_CHANGES.html file attachment.  The left side is before the changes and right side the after changes.

This result in the results shown in the hfs.txt file attachment.

[DL Wicksell]

You are most welcome. 

--

David Wicksell
Fourth Watch Software LC

--
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+u...@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "Hardhats" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/da9352d3-cc17-45fa-81b9-d4023503f0dbn%40googlegroups.com.

[Rob Kellock]

Nice.  I've done the same fix here.  Added in a few more spaces though so that the Page Number is right aligned with the ------- on the line below.

[Rob Kellock]

So Ben, I pulled down your FileMan with Printing with the idea that I would take your source code ONLY and BUILD UP from there. I've tried to learn how things work in the past, by stripping code out and I get so lost in the complexities in large systems that I achieve little doing it that way. I much prefer starting with nothing and adding to it so that I understand each step.

Pretty soon I hit a snag with the message:

HOME DEVICE (/dev/pts/0) DOES NOT EXIST IN THE DEVICE FILE
PLEASE CONTACT YOUR SYSTEM MANAGER!

It took me ages to work out that means I had no #3.5 DEVICE lurking in the global ^DIC(3.5,0)="DEVICE^3.5" or at least I think that was where it was supposed to be. I'm still unsure just where #3.5 DEVICE actually resides.  I assume it's in a global somewhere? As DINIT hadn't placed it there I guessed it was in your database. Eventually, I connected to your database and discover there are scores of devices! I'm not sure what happens when you install the Kernel, but I have a strong feeling it won't do any interrogation of my network to plug and play to my printer. Instead, I will have to hope that a compatible machine is listed and use that. I think that is very unlikely because it is a Windows printer which I talk to using CUPS (I'm on Ubuntu 20.04.4 LTS). So... I'm not quite sure where to go from here? As a start I was hoping for a wider screen display beyond 80 columns, but I haven't worked out how to do that either.

Just wanted you to know that I have used your code and it does work, mostly, I think, but I'm unsure how to use it right now. Time to go to bed.....!

Cheers,

Rob.

[Rob Kellock]

I've just realised what the ^DIC global is doing. It's quite literally a dictionary that tells you where everything else is stored. Cool!

[Benjamin Irwin]

Yes, and the ^DD global stores all of the file and field definitions.

If you type:  D P^DI at the MUMPS prompt that should take you to a OPTION prompt.  Two question marks there will list OPTIONS that you can use to work with the FileMan database.

fileman>D P^DI

VA FileMan 22.2

Select OPTION: ??

    

   Choose from:

   1            ENTER OR EDIT FILE ENTRIES

   2            PRINT FILE ENTRIES

   3            SEARCH FILE ENTRIES

   4            MODIFY FILE ATTRIBUTES

   5            INQUIRE TO FILE ENTRIES

   6            UTILITY FUNCTIONS

   7            OTHER OPTIONS

   8            DATA DICTIONARY UTILITIES

   9            TRANSFER ENTRIES

   

Select OPTION: 

Take a look at the manuals to work through the options.  They are there to edit the database and enter/edit the data within the database.  With the almost clean database there aren't any non-system databases yet.  You can look at the structure of the DEVICE and TERMINAL TYPE files for some examples.

I think you have already found this:  Using #8 then #1 you can show the database structures.

fileman>D P^DI

VA FileMan 22.2

Select OPTION: 8  DATA DICTIONARY UTILITIES

Select DATA DICTIONARY UTILITY OPTION: 1  LIST FILE ATTRIBUTES

 START WITH What File: DEVICE//           (53 entries)

      GO TO What File: DEVICE//           (53 entries)

      Select SUB-FILE: 

Select LISTING FORMAT: STANDARD//   

Start with field: FIRST// 

DEVICE: HOME//   TELNET

STANDARD DATA DICTIONARY #3.5 -- DEVICE FILE                  6/20/24    PAGE 1

STORED IN ^%ZIS(1,  (53 ENTRIES)   SITE: WWW.BMIRWIN.COM   UCI: VISTA,VISTA (VER

SION 8.0)   

DATA          NAME                  GLOBAL        DATA

ELEMENT       TITLE                 LOCATION      TYPE

-------------------------------------------------------------------------------

This file defines all input/output devices that can be accessed from this CPU

(definitions are not account-specific).  Each device is identified with a

unique name.  Each is associated with a $I value which may correspond with a

hardware port or, on layered systems, a host file or directory.  If there are

several devices for the same volume set and $I, one may be given sign-on system

status.  Devices may also be assigned to hunt groups to share work.  This file

is cross-referenced by name, $I, volume set(CPU), and sign-on/system device. 

It is also cross-referenced by hunt group, local synonym, mnemonic, subtype,

and form currently mounted.  

[Benjamin Irwin]

The following link is a web representation of the dictionary definitions of the full VistA system.

https://edu2.opensourcevista.net/vista/dictionary.php

Clicking on the dictionary name produces a "STANDARD" dictionary listing.

Clicking on the global produces a "GLOBAL MAP" dictionary listing.

[Rob Kellock]

Now that is exactly what I wanted :-)  Thanks, Ben. By the way is that a live web site looking into a VistA instance? By now I imagine some parts of VistA are accessed using web interfaces.  No doubt there's several different ways of doing that.  But one step at a time!  I need to completely understand FileMan before getting ahead of myself.

I do appreciate all the help I'm receiving from this list.

Cheers,

Rob.

[Benjamin Irwin]

Rob,

Yes, this information is interactive with a VistA instance.  It gets updated monthly with the release of the FOIA patches from the VA.  Most of the information is directly drawn from the VistA instance using PHP and Node JS.  The dictionary information required a manual update script to be run.

These two web pages show how the information is retrieved with some helpful routines and coding.

https://edu2.opensourcevista.net/vista/index.php

https://edu2.opensourcevista.net/vista/index.pdf

The index page was me learning what could be done with the markdown language.  It is written in a text markdown document and automatically translated to html through Pandoc.  I guess it's one way of doing the documentation, but nothing exciting.

Ben