Yottadb help needed

[Kevin Toppenberg]

I want to rebuild my VistA server, which is currently running on Ubuntu 2014 LTS.  I tried following Ubuntu's upgrade path, but it caused problems I couldn't recover from and I had to revert to a backup.

So I am going to try to start fresh with Ubuntu 20.04 LTS, I would like to take this opportunity to change from GT.M to YottaDb.  My current GT.M is version 6.2002A.

I have reviewed the YottaDB website (https://yottadb.com/resources/documentation/).  Apparently r1.0 is a drop-in replacement for GT.M 6.3001A.  (https://gitlab.com/YottaDB/DB/YDB/-/tags/r1.00)

Questions:

• Is there a document giving guidance on an GT.M -> upgrade path?
• Can I go directly from GT.M 6.2002A to the latest version of YottaDB?  If not, what intermediate steps should I plan on?
  
• It looks like in GT.M encrypting the database was optional.  The YottaDB installation guide (https://docs.yottadb.com/AdminOpsGuide/installydb.html) seems to indicated that this is standard now.  Is this true?  If so, how much of a performance hit does this encryption effect?
• Is there a better place to be asking these questions?
• Anything else I should know / consider?
  

Thanks in advance

Kevin

[Sam Habiel]

> Is there a document giving guidance on an GT.M -> upgrade path?

Upgrade instructions are provided for each release. For example:
https://gitlab.com/YottaDB/DB/YDB/-/tags/r1.30#upgrading-to-yottadb-r130.
Do use the latest version; not 1.0.

> Can I go directly from GT.M 6.2002A to the latest version of YottaDB? If not, what intermediate steps should I plan on?

Yes

>It looks like in GT.M encrypting the database was optional. The YottaDB installation guide (https://docs.yottadb.com/AdminOpsGuide/installydb.html) seems to indicated that this is standard now. Is this true? If so, how much of a performance hit does this encryption effect?

Encryption is optional

>Is there a better place to be asking these questions?

Customers can get support directly from the company. But here is best for now.

>Anything else I should know / consider?

You can keep using your $gtm variables without change; but we now have
an SQL data later that goes on top of VistA called Octo. If you want
to use that, you need to switch to $yottadb variables.

> --
> --
> http://groups.google.com/group/Hardhats
> To unsubscribe, send email to Hardhats+u...@googlegroups.com
>
> ---
> You received this message because you are subscribed to the Google Groups "Hardhats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/9666d53d-00f9-40cc-b8a1-72a97e504531n%40googlegroups.com.

[Kevin Toppenberg]

Thanks Sam!

Kevin

[Kevin Toppenberg]

A follow up question:

Which is the best directory to install YottaDB into?

My current environment has the following setup:

gtm is stored in /opt/lsb_gtm

and my vista system is in

/opt/worldvista/EHR/

But it seems I have heard in years past, that there would be better areas to store this.

I have been doing some linux reading, and found this:

The /opt directory contains subdirectories for optional software packages. It’s commonly used by proprietary software that doesn’t obey the standard file system hierarchy – for example, a proprietary program might dump its files in /opt/application when you install it.

And regarding /usr/bin/

/usr – User Binaries & Read-Only Data

The /usr directory contains applications and files used by users, as opposed to applications and files used by the system. For example, non-essential applications are located inside the /usr/bin directory instead of the /bin directory and non-essential system administration binaries are located in the /usr/sbin directory instead of the /sbin directory. Libraries for each are located inside the /usr/lib directory. The /usr directory also contains other directories – for example, architecture-independent files like graphics are located in /usr/share.

The /usr/local directory is where locally compiled applications install to by default – this prevents them from mucking up the rest of the system.

So as I construct a new server, any thoughts about best place to install YottaDB?

Thanks

Kevin

[Kevin Toppenberg]

I got impatient and went with the default of /opt/yottadb

I am following the installation guide here:

https://docs.yottadb.com/AdminOpsGuide/installydb.html#installing-yottadb

I was able to run the ydbinstall.sh script.  I had to install one dependency, but the script gave me full instructions to do so, which worked.

The next step in the installation is for installing the "Reference Implementation Plugin".  I have no idea what a "Reference Implementation Plugin" is, but it seems to be the next recommended step. 

The guide gives the following example command:

sudo apt-get install libgcrypt11-dev libgpgme11-dev libconfig-dev libssl-dev

I am having problems with installing libgcrypt11 as below

kdt0p@ubuntu64:/opt/yottadb$ sudo apt update
Hit:1 http://us.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
kdt0p@ubuntu64:/opt/yottadb$ sudo apt-get install -y libgcrypt11-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package libgcrypt11-dev

So I tried installing just plain libgcrypt-dev,   i.e. removing the "11".  This ends up installing libgcrypt20.  Will this be OK?

kdt0p@ubuntu64:/opt/yottadb$ sudo apt-get install -y libgcrypt-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libgcrypt20-dev' instead of 'libgcrypt-dev'
The following packages were automatically installed and are no longer required:
  linux-headers-5.8.0-43-generic linux-hwe-5.8-headers-5.8.0-43
  linux-image-5.8.0-43-generic linux-modules-5.8.0-43-generic
  linux-modules-extra-5.8.0-43-generic
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libgpg-error-dev
Suggested packages:
  libgcrypt20-doc
The following NEW packages will be installed:
  libgcrypt20-dev libgpg-error-dev
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 579 kB of archives.
After this operation, 2,522 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libgpg-error-dev amd64 1.37-1 [109 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libgcrypt20-dev amd64 1.8.5-5ubuntu1 [470 kB]
Fetched 579 kB in 1s (1,035 kB/s)    
Selecting previously unselected package libgpg-error-dev.
(Reading database ... 206176 files and directories currently installed.)
Preparing to unpack .../libgpg-error-dev_1.37-1_amd64.deb ...
Unpacking libgpg-error-dev (1.37-1) ...
Selecting previously unselected package libgcrypt20-dev.
Preparing to unpack .../libgcrypt20-dev_1.8.5-5ubuntu1_amd64.deb ...
Unpacking libgcrypt20-dev (1.8.5-5ubuntu1) ...
Setting up libgpg-error-dev (1.37-1) ...
Setting up libgcrypt20-dev (1.8.5-5ubuntu1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for install-info (6.7.0.dfsg.2-5) ...
kdt0p@ubuntu64:/opt/yottadb$ sudo apt-get install -y libgcrypt-dev

I was also to also get libgpgme installed with this:  

     sudo apt-get install -y libgpgme-dev

and libconfig with:

     sudo apt-get install -y libconfig-dev

and libssl with

     sudo apt-get install -y libssl-dev

QUESTIONS:

1) What is Reference Implementation Plugin

2) as above, is libgcrypt20 an OK substitution for libgcrypt11?

3) I don't plan on using encryption of the database.  Did I need to install these libgcrypt etc packages in my case?

Thanks

Kevin

[K.S. Bhaskar]

Kevin –

Are you attempting to install the packages needed to develop YottaDB, i.e., to compile it from source code? In any case, if you're not planning to use database encryption or to use M support for TCP connections with TLS, you don't need the encryption plugin (also called the Reference Implementation Plugin). To be able to run VistA on YottaDB, do the following (from https://yottadb.com/product/get-started/):

Download the YottaDB installer into a temporary directory: mkdir /tmp/tmp ; wget -P /tmp/tmp https://gitlab.com/YottaDB/DB/YDB/raw/master/sr_unix/ydbinstall.sh

Make the file executable: cd /tmp/tmp ; chmod +x ydbinstall.sh

Install YottaDB: sudo ./ydbinstall.sh --utf8 default --verbose

The web page lists more steps if you are a new user, but the above should suffice since you are already using GT.M and for VistA YottaDB is effectively the same as GT.M. The above will install YottaDB in /usr/local/lib/yottadb/r130. If you want to install it elsewhere use the --installdir option of ydbinstall.sh. ./ydbinstall.sh --help will give you a list of its options.

Regards

– Bhaskar

[Kevin Toppenberg]

Bhaskar,

Thank you for your reply.  I was following the install instructions in the Yottadb Admin and Operations guide (https://docs.yottadb.com/AdminOpsGuide/installydb.html#installing-yottadb).

I will take a look at the other location you indicated: https://yottadb.com/product/get-started/

Thanks!

Kevin

[Arthur Ingram]

Hello

Have you looked at 

https://hardhats.org/projects/PROJECTSmain.html#Installing_VistA  documentation ?

or suggestions mention in recent VAPALS-ELCAP Management System file layout at https://github.com/VA-PALS-ELCAP/SAMI-VAPALS-ELCAP

art

[Kevin Toppenberg]

Arthur,

Thanks for those links.  I will check them out.

Kevin

[Kevin Toppenberg]

OK.  I am finally getting back to work on this some more, and having problems...

I have tried to change all the links from the older version of GTM to my new yottadb.

I am able to launch into yottadb and get a mumps prompt.

When I try to launch VistA via "do ^XUP", I get the following error.

%YDB-E-GDINVALID, Unrecognized Global Directory file format: /opt/worldvista/EHR/g/mumps.gld, expected label: GTCGBDUNX112, found: GTCGBDUNX010

I suspect I need to update the .gld file with the DSE for GDE utility.  It has been so many years ago that I can't remember how to do this.

I am trying to figure this out in the Acculturation guide here:   https://docs.yottadb.net/AcculturationGuide/acculturation.html#global-directories-point-to-global-variables

I see this command:  yottadb -run GDE

The problem is, when I do a "DO ^GDE" from my yottadb prompt, I get a file not found error:

ASTRON>do ^GDE    
%YDB-E-ZLINKFILE, Error while zlinking "GDE"                                                                                    
%YDB-E-FILENOTFND, File GDE not found

In my old (correctly running) VistA system, I can find a GDE.o file in my gtm directory, but no GDE.m file that I might copy to the new. 

In my new yottadb system there is nothing about GDE,

kdt0p@ubuntu64:~$ cd $gtm_dist                                                                                                  
kdt0p@ubuntu64:/opt/worldvista/EHR/m$ ls -Al GDE*  

ls: cannot access 'GDE*': No such file or directory

FYI, Below is the script I use to launch my mumps / VistA system

#!/bin/bash
#Modified 5/5/2021  K. Toppenberg

echo " "
echo "YottaDB VistA Startup Script"

. /opt/worldvista/EHR/bin/setup_env
export EDITOR=$(which joe)
#export TERM=ansi

rm -f ~/*.mj[eo]

echo "Entering YottaDB system now... "
echo "BLASTING OFF......"
echo " WITH"
echo "  ASTRONAUT"
echo "   VISTA!"
#-----------------------------

#stty susp \000

if [ $# -gt 0 ]
  then
    echo "Automatically launching program: $1"
    $gtm_dist/mumps -r $1
  else
    $gtm_dist/mumps -dir
fi

echo " "
echo "Leaving YottaDB, returning to Linux..."
echo " "

===================================

Below is the environment script linked above /opt/worldvista/EHR/bin/setup_env

#!/bin/sh
## Modified 5/5/21 K. Toppenberg -- 5/5/21

echo "---Starting Setup_env script---"

#***********************************************************
#***********************************************************
# NOTICE
# The startup script that is launched from xinet.d does NOT
# appear to use this script, and instead uses:
#   /opt/worldvista/EHR/etc/env
#***********************************************************
#***********************************************************

export GTM_REPLICATION=off
export vista_home="/opt/worldvista/EHR"
$vista_home/bin/env

echo "vista_home="$vista_home
export VH=${vista_home} # a short name for paths, etc - temp
echo "VH="$VH
export gtm_dist="${VH}/m"
echo "gtm_dist="$gtm_dist
export gtm_prompt="yottadb>"

#//kt  removing 5/5/21
#export node_home="/opt/ewdlite/node_modules/nodem"
#export GTMCI="${node_home}/resources/calltab.ci"

#export gtm_sysid="xxxx"
export gtm_log="${VH}/log"
export gtmgbldir="${VH}/g/mumps.gld"

#//kt 5/5/21  export gtmroutines="${VH}/o(${VH}/p ${VH}/r ${node_home}/src)"
export gtmroutines="${VH}/o(${VH}/p ${VH}/r)"
export gtmroutines="${gtmroutines} ${gtm_dist}"
export gtm_zinterrupt='I $$JOBEXAM^ZU($ZPOSITION)'
# MD5 Library external-call table
#export GTMXC_md5="${VH}/w/xc/gtm_md5.xc"
##
export PATH="${VH}/m:${PATH}"
unset  VH

# Define command aliases
alias GTM="${gtm_dist}/mumps -direct"
alias gtm="${gtm_dist}/mumps -direct"
alias mupip="${gtm_dist}/mupip"
alias gde="${gtm_dist}/mumps -run ^GDE"
alias GDE="${gtm_dist}/mumps -run ^GDE"
alias lke="${gtm_dist}/lke"
alias dse="${gtm_dist}/dse"
alias LKE="${gtm_dist}/lke"
alias DSE="${gtm_dist}/dse"
alias rundown="${gtm_dist}/mupip rundown -r \"*\""

#Required due to GT.M change as of 6.0-002A
export gtm_boolean=1
export gtm_side_effects=1
export gtm_zquit_anyway=1

#//kt 5/5/21  export GTMCI="/home/kdt0p/qewd/node_modules/nodem/resources/nodem.ci"
echo "---Done with Setup_env script---"

===================================

Questions

1) Am I correct that I need to use GDE to get my .gld file up to newer version?

2) Why is GDE missing?

3) How do I fix this 'Unrecognized Global Directory file format'  error?

Thanks

Kevin T

[Sam Habiel]

1) Am I correct that I need to use GDE to get my .gld file up to newer version?

Yes. That's correct.

2) Why is GDE missing?

I suspect you are missing $gtm_dist/libgtmutil.so on your
$gtmroutines. Your scripts were written long ago for x86 systems
probably, when these .sos weren't a thing that existed yet. See here
for more details:
https://www.hardhats.org/projects/New/InstallVistAOnGTM.html

3) How do I fix this 'Unrecognized Global Directory file format' error?

Just type "exit" when you are inside GDE. The directory format will be
updated. In rare circumstances (if you are switching to a different
architecture) it may not work, and you have to recreate your GDE from
your original definitions. Let's not go there until we need to.

--Sam

> --
> --
> http://groups.google.com/group/Hardhats
> To unsubscribe, send email to Hardhats+u...@googlegroups.com
>
> ---
> You received this message because you are subscribed to the Google Groups "Hardhats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+u...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/8be7fb7e-c14c-488d-bb16-ded0e3ecf483n%40googlegroups.com.

[Kevin Toppenberg]

Sam,

Thank you for your reply.

Here is my $gtmroutines variable.

kdt0p@ubuntu64:~$ echo $gtmroutines

/opt/worldvista/EHR/o(/opt/worldvista/EHR/p /opt/worldvista/EHR/r) /opt/worldvista/EHR/m

and here is a full directory listing of my /opt/worldvista/EHR/m/ folder:

total 12956
-r--r--r-- 1 kdt0p kdt0p    3115 Aug 11  2020 _convbaseutil.m
-r--r--r-- 1 kdt0p kdt0p    3115 Aug 11  2020 _CONVBASEUTIL.m
-r--r--r-- 1 kdt0p kdt0p   34520 Apr 10  2020 COPYING
-r--r--r-- 1 kdt0p kdt0p     500 Apr 10  2020 custom_errors_sample.txt
-r--r--r-- 1 kdt0p kdt0p    2858 Aug 11  2020 _date.m
-r--r--r-- 1 kdt0p kdt0p    2858 Aug 11  2020 _DATE.m
-r--r--r-- 1 kdt0p kdt0p   11271 Aug 11  2020 decomment.m
-r--r--r-- 1 kdt0p kdt0p   11271 Aug 11  2020 DECOMMENT.m
-r--r--r-- 1 kdt0p kdt0p    2666 Aug 11  2020 _dh.m
-r--r--r-- 1 kdt0p kdt0p    2666 Aug 11  2020 _DH.m
-r--r--r-- 1 kdt0p kdt0p     849 Aug 11  2020 _d.m
-r--r--r-- 1 kdt0p kdt0p     849 Aug 11  2020 _D.m
-r--r--r-- 1 kdt0p kdt0p    1607 Aug 11  2020 _do.m
-r--r--r-- 1 kdt0p kdt0p    1607 Aug 11  2020 _DO.m
-r-xr-xr-x 1 kdt0p kdt0p   13008 Apr 19 12:24 dse
-r--r--r-- 1 kdt0p kdt0p  471040 Aug 11  2020 dsehelp.dat
-r--r--r-- 1 kdt0p kdt0p    2560 Aug 11  2020 dsehelp.gld
-r--r--r-- 1 kdt0p kdt0p   11822 Aug 11  2020 _dsewrap.m
-r--r--r-- 1 kdt0p kdt0p   11822 Aug 11  2020 _DSEWRAP.m
-r--r--r-- 1 kdt0p kdt0p    7138 Aug 11  2020 _dumpfhead.m
-r--r--r-- 1 kdt0p kdt0p    7138 Aug 11  2020 _DUMPFHEAD.m
-r--r--r-- 1 kdt0p kdt0p     822 Aug 11  2020 _exp.m
-r--r--r-- 1 kdt0p kdt0p     822 Aug 11  2020 _EXP.m
-r--r--r-- 1 kdt0p kdt0p     533 Aug 11  2020 _fl.m
-r--r--r-- 1 kdt0p kdt0p     533 Aug 11  2020 _FL.m
-r--r--r-- 1 kdt0p kdt0p    1091 Aug 11  2020 _freecnt.m
-r--r--r-- 1 kdt0p kdt0p    1091 Aug 11  2020 _FREECNT.m
-r-xr-xr-x 1 kdt0p kdt0p   13016 Apr 19 12:24 ftok
-r--r--r-- 1 kdt0p kdt0p    3234 Aug 11  2020 _gbldef.m
-r--r--r-- 1 kdt0p kdt0p    3234 Aug 11  2020 _GBLDEF.m
-r--r--r-- 1 kdt0p kdt0p    2670 Aug 11  2020 _gce.m
-r--r--r-- 1 kdt0p kdt0p    2670 Aug 11  2020 _GCE.m
-r--r--r-- 1 kdt0p kdt0p    3010 Aug 11  2020 _gc.m
-r--r--r-- 1 kdt0p kdt0p    3010 Aug 11  2020 _GC.m
-r--r--r-- 1 kdt0p kdt0p     565 Apr 19 12:24 gdedefaults
-r--r--r-- 1 kdt0p kdt0p  471040 Aug 11  2020 gdehelp.dat
-r--r--r-- 1 kdt0p kdt0p    2560 Aug 11  2020 gdehelp.gld
-r--r--r-- 1 kdt0p kdt0p     772 Aug 11  2020 _gd.m
-r--r--r-- 1 kdt0p kdt0p     772 Aug 11  2020 _GD.m

-r--r--r-- 1 kdt0p kdt0p    5940 Aug 11  2020 _ged.m
-r--r--r-- 1 kdt0p kdt0p    5940 Aug 11  2020 _GED.m
-r--r--r-- 1 kdt0p kdt0p    4540 Aug 11  2020 _gi.m
-r--r--r-- 1 kdt0p kdt0p    4540 Aug 11  2020 _GI.m
-r--r--r-- 1 kdt0p kdt0p    2358 Aug 11  2020 _g.m
-r--r--r-- 1 kdt0p kdt0p    2358 Aug 11  2020 _G.m
-r--r--r-- 1 kdt0p kdt0p    2996 Aug 11  2020 _go.m
-r--r--r-- 1 kdt0p kdt0p    2996 Aug 11  2020 _GO.m
-r--r--r-- 1 kdt0p kdt0p    6362 Aug 11  2020 _gsel.m
-r--r--r-- 1 kdt0p kdt0p    6362 Aug 11  2020 _GSEL.m
-r--r--r-- 1 kdt0p kdt0p    3000 Aug 11  2020 _gse.m
-r--r--r-- 1 kdt0p kdt0p    3000 Aug 11  2020 _GSE.m
-r-xr-xr-x 1 kdt0p kdt0p   13024 Apr 19 12:24 gtcm_gnp_server
-r-xr-xr-x 1 kdt0p kdt0p   13608 Apr 19 12:24 gtcm_pkdisp
-r-xr-xr-x 1 kdt0p kdt0p   13016 Apr 19 12:24 gtcm_play
-r-xr--r-- 1 kdt0p kdt0p    3689 Apr 19 12:24 gtcm_run
-r-xr-xr-x 1 kdt0p kdt0p   13016 Apr 19 12:24 gtcm_server
-r-xr-xr-x 1 kdt0p kdt0p   13024 Apr 19 12:24 gtcm_shmclean
-r-xr--r-- 1 kdt0p kdt0p     957 Apr 19 12:24 gtcm_slist
lrwxrwxrwx 1 kdt0p kdt0p       5 Apr 19 11:47 gtm -> ./ydb
-r--r--r-- 1 kdt0p kdt0p    6635 Jul 13  2020 gtm_common_defs.h
lrwxrwxrwx 1 kdt0p kdt0p      22 Apr 19 11:47 gtmcrypt_interface.h -> ./ydbcrypt_interface.h
-r--r--r-- 1 kdt0p kdt0p    1786 Apr 10  2020 gtm_descript.h
-r--r--r-- 1 kdt0p kdt0p     217 Jul 13  2020 gtmgblstat.xc
-r--r--r-- 1 kdt0p kdt0p 2727936 Aug 11  2020 gtmhelp.dat
-r--r--r-- 1 kdt0p kdt0p    2560 Aug 11  2020 gtmhelp.gld
-r--r--r-- 1 kdt0p kdt0p    7363 Aug 11  2020 gtmhelp.m
-r--r--r-- 1 kdt0p kdt0p    7363 Aug 11  2020 GTMHELP.m
-rw-rw-r-- 1 kdt0p kdt0p   23000 May  5 11:53 GTMHELP.o
-r--r--r-- 1 kdt0p kdt0p    3797 Jul 13  2020 gtm_limits.h
lrwxrwxrwx 1 kdt0p kdt0p      13 Apr 19 11:47 gtmprofile -> ./ydb_env_set
-r-xr-xr-x 1 kdt0p kdt0p   24736 Apr 19 12:24 gtmsecshr
dr-x------ 2 kdt0p kdt0p    4096 Apr 19 12:24 gtmsecshrdir
-r--r--r-- 1 kdt0p kdt0p    2505 Apr 10  2020 gtm_sizeof.h
-r--r--r-- 1 kdt0p kdt0p    6559 Aug  1  2020 gtm_stdio.h
-r--r--r-- 1 kdt0p kdt0p    2307 Jul 13  2020 gtm_stdlib.h
-r--r--r-- 1 kdt0p kdt0p    3234 Jul 13  2020 gtm_string.h
-r--r--r-- 1 kdt0p kdt0p     848 Apr 10  2020 gtm_strings.h
lrwxrwxrwx 1 kdt0p kdt0p      21 Apr 19 11:47 gtm_tls_interface.h -> ./ydb_tls_interface.h
-r--r--r-- 1 kdt0p kdt0p    4542 Aug 11  2020 gtmxc_types.h

-r--r--r-- 1 kdt0p kdt0p    4542 Aug 11  2020 gtmxc_types.h
-r--r--r-- 1 kdt0p kdt0p    2223 Aug 11  2020 _hd.m
-r--r--r-- 1 kdt0p kdt0p    2223 Aug 11  2020 _HD.m
-r--r--r-- 1 kdt0p kdt0p    1227 Aug 11  2020 _hex2utf.m
-r--r--r-- 1 kdt0p kdt0p    1227 Aug 11  2020 _HEX2UTF.m
-r--r--r-- 1 kdt0p kdt0p    1879 Aug 11  2020 _h.m
-r--r--r-- 1 kdt0p kdt0p    1879 Aug 11  2020 _H.m
-r--r--r-- 1 kdt0p kdt0p    1078 Aug 11  2020 _ho.m
-r--r--r-- 1 kdt0p kdt0p    1078 Aug 11  2020 _HO.m
-r--r--r-- 1 kdt0p kdt0p    1255 Aug 11  2020 _lcase.m
-r--r--r-- 1 kdt0p kdt0p    1255 Aug 11  2020 _LCASE.m
-r--r--r-- 1 kdt0p kdt0p     992 Aug 11  2020 _lclcol.m
-r--r--r-- 1 kdt0p kdt0p     992 Aug 11  2020 _LCLCOL.m
lrwxrwxrwx 1 kdt0p kdt0p      15 Apr 19 11:47 libgtmshr.so -> ./libyottadb.so
lrwxrwxrwx 1 kdt0p kdt0p      19 Apr 19 11:47 libgtmutil.so -> ./libyottadbutil.so
-r--r--r-- 1 kdt0p kdt0p    2172 Aug  5  2020 libydberrors2.h
-r--r--r-- 1 kdt0p kdt0p   59750 Jul 13  2020 libydberrors.h
-r--r--r-- 1 kdt0p kdt0p   22465 Aug 11  2020 libyottadb.h
-r-xr-xr-x 1 kdt0p kdt0p 4710432 Apr 19 12:24 libyottadb.so
-r-xr-xr-x 1 kdt0p kdt0p 2208648 Apr 19 12:24 libyottadbutil.so
-r-xr-xr-x 1 kdt0p kdt0p   13008 Apr 19 12:24 lke
-r--r--r-- 1 kdt0p kdt0p  471040 Aug 11  2020 lkehelp.dat
-r--r--r-- 1 kdt0p kdt0p    2560 Aug 11  2020 lkehelp.gld
-r--r--r-- 1 kdt0p kdt0p     856 Apr 10  2020 main_pragma.h
-r--r--r-- 1 kdt0p kdt0p    3646 Jul 13  2020 mmrhash.h
-r--r--r-- 1 kdt0p kdt0p    1884 Aug 11  2020 _mpiece.m
-r--r--r-- 1 kdt0p kdt0p    1884 Aug 11  2020 _MPIECE.m
lrwxrwxrwx 1 kdt0p kdt0p       9 Apr 19 11:47 mumps -> ./yottadb
-r-xr-xr-x 1 kdt0p kdt0p   13016 Apr 19 12:24 mupip
-r--r--r-- 1 kdt0p kdt0p  880640 Aug 11  2020 mupiphelp.dat
-r--r--r-- 1 kdt0p kdt0p    2560 Aug 11  2020 mupiphelp.gld
-r--r--r-- 1 kdt0p kdt0p    1060 Aug 11  2020 _od.m
-r--r--r-- 1 kdt0p kdt0p    1060 Aug 11  2020 _OD.m
-r--r--r-- 1 kdt0p kdt0p    1204 Aug 11  2020 _oh.m
-r--r--r-- 1 kdt0p kdt0p    1204 Aug 11  2020 _OH.m
-r--r--r-- 1 kdt0p kdt0p     687 Aug 11  2020 _patcode.m
-r--r--r-- 1 kdt0p kdt0p     687 Aug 11  2020 _PATCODE.m
-r--r--r-- 1 kdt0p kdt0p    9428 Aug 11  2020 _peekbyname.m
-r--r--r-- 1 kdt0p kdt0p    9428 Aug 11  2020 _PEEKBYNAME.m

dr-xr-xr-x 5 kdt0p kdt0p    4096 Apr 19 11:48 plugin
-r--r--r-- 1 kdt0p kdt0p    2286 Aug 11  2020 _randstr.m
-r--r--r-- 1 kdt0p kdt0p    2286 Aug 11  2020 _RANDSTR.m
-r--r--r-- 1 kdt0p kdt0p    6154 Aug 11  2020 _rce.m
-r--r--r-- 1 kdt0p kdt0p    6154 Aug 11  2020 _RCE.m
-r--r--r-- 1 kdt0p kdt0p     771 Aug 11  2020 _rd.m
-r--r--r-- 1 kdt0p kdt0p     771 Aug 11  2020 _RD.m
-r--r--r-- 1 kdt0p kdt0p    9194 Aug 11  2020 README.md
-r--r--r-- 1 kdt0p kdt0p    3918 Aug 11  2020 _ri.m
-r--r--r-- 1 kdt0p kdt0p    3918 Aug 11  2020 _RI.m
-r--r--r-- 1 kdt0p kdt0p    5055 Aug 11  2020 _ro.m
-r--r--r-- 1 kdt0p kdt0p    5055 Aug 11  2020 _RO.m
-r--r--r-- 1 kdt0p kdt0p    9561 Aug 11  2020 _rsel.m
-r--r--r-- 1 kdt0p kdt0p    9561 Aug 11  2020 _RSEL.m
-r--r--r-- 1 kdt0p kdt0p    1383 Aug 11  2020 _rse.m
-r--r--r-- 1 kdt0p kdt0p    1383 Aug 11  2020 _RSE.m
-r--r--r-- 1 kdt0p kdt0p   65208 Aug 11  2020 scantypedefs.m
-r--r--r-- 1 kdt0p kdt0p   65208 Aug 11  2020 SCANTYPEDEFS.m
-r-xr-xr-x 1 kdt0p kdt0p   12864 Apr 19 12:24 semstat2
-r--r--r-- 1 kdt0p kdt0p     822 Aug 11  2020 _sqroot.m
-r--r--r-- 1 kdt0p kdt0p     822 Aug 11  2020 _SQROOT.m
-r--r--r-- 1 kdt0p kdt0p    3028 Aug 11  2020 _ti.m
-r--r--r-- 1 kdt0p kdt0p    3028 Aug 11  2020 _TI.m
-r--r--r-- 1 kdt0p kdt0p     793 Aug 11  2020 _t.m
-r--r--r-- 1 kdt0p kdt0p     793 Aug 11  2020 _T.m
-r--r--r-- 1 kdt0p kdt0p     755 Aug 11  2020 _to.m
-r--r--r-- 1 kdt0p kdt0p     755 Aug 11  2020 _TO.m
-r--r--r-- 1 kdt0p kdt0p     933 Aug 11  2020 _trim.m
-r--r--r-- 1 kdt0p kdt0p     933 Aug 11  2020 _TRIM.m
-r--r--r-- 1 kdt0p kdt0p    1255 Aug 11  2020 _ucase.m
-r--r--r-- 1 kdt0p kdt0p    1255 Aug 11  2020 _UCASE.m
-r--r--r-- 1 kdt0p kdt0p    1087 Aug 11  2020 _utf2hex.m
-r--r--r-- 1 kdt0p kdt0p    1087 Aug 11  2020 _UTF2HEX.m
-r--r--r-- 1 kdt0p kdt0p    3766 Aug 11  2020 _xcmd.m
-r--r--r-- 1 kdt0p kdt0p    3766 Aug 11  2020 _XCMD.m
-rw-rw-r-- 1 kdt0p kdt0p    9496 May  5 11:53 _XCMD.o
-r-xr-xr-x 1 kdt0p kdt0p    1152 Apr 19 12:24 ydb
-r--r--r-- 1 kdt0p kdt0p    9527 Jul 13  2020 ydbcrypt_interface.h
-r--r--r-- 1 kdt0p kdt0p   33299 Aug 11  2020 _ydbenv.m

-r--r--r-- 1 kdt0p kdt0p   33299 Aug 11  2020 _YDBENV.m
-r-xr-xr-x 1 kdt0p kdt0p    3734 Apr 19 12:24 ydb_env_set
-r-xr-xr-x 1 kdt0p kdt0p    1279 Apr 19 12:24 ydb_env_unset
-r--r--r-- 1 kdt0p kdt0p    4083 Aug 11  2020 _ydbprocstuckexec.m
-r--r--r-- 1 kdt0p kdt0p    4083 Aug 11  2020 _YDBPROCSTUCKEXEC.m
-r--r--r-- 1 kdt0p kdt0p   19778 Jul 13  2020 ydb_tls_interface.h
-r--r--r-- 1 kdt0p kdt0p   11124 Aug 11  2020 _ygblstat.m
-r--r--r-- 1 kdt0p kdt0p   11124 Aug 11  2020 _YGBLSTAT.m
-r-xr-xr-x 1 kdt0p kdt0p   13016 Apr 19 12:24 yottadb
-r--r--r-- 1 kdt0p kdt0p     242 Apr 19 12:24 yottadb.pc
-r--r--r-- 1 kdt0p kdt0p   10706 Aug 11  2020 _zshowvtolcl.m
-r--r--r-- 1 kdt0p kdt0p   10706 Aug 11  2020 _ZSHOWVTOLCL.m

So the files you mention seem to be there.

But I am still confused what libgtmutil.so has to do with GDE.  It seems that there is a source code file (e.g. a .m file) missing. 

kdt0p@ubuntu64:~$ yottadb -run GDE

%YDB-E-ZLINKFILE, Error while zlinking "GDE"
%YDB-E-FILENOTFND, File GDE not found

Doesn't this indicate that GDE.m is missing?

Thanks again

Kevin

[Sam Habiel]

There is complexity and history in many aspects.

GDE is actually packaged inside the shared library and the source code
deleted from distribution, to make things small, as it's a giant
program.

That's why you don't see it.

--Sam

> To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/0191b335-fa55-4fcc-89c0-96e9f19e58ban%40googlegroups.com.

[Kevin Toppenberg]

I suspect you are missing $gtm_dist/libgtmutil.so on your
$gtmroutines. Your scripts were written long ago for x86 systems
probably, when these .sos weren't a thing that existed yet. See here
for more details:
https://www.hardhats.org/projects/New/InstallVistAOnGTM.html

I misunderstood what you were saying.  Reading that linked document helped me understand.

# Where the routines are. # If you run 32 bit GT.M/YottaDB, you need to remove /libgtmutil.so

# On older versions of GT.M (<6.2), the * isn't recognized.

# There should be no reason for you to run 32-bit GT.M these days.

export gtmroutines="${vista_home}/o*(${vista_home}/r) $gtm_dist/libgtmutil.so"

After making the change above, my $zroutines looks like this.

yottadb>w $zroutines
/opt/worldvista/EHR/o*(/opt/worldvista/EHR/p /opt/worldvista/EHR/r) /opt/worldvista/EHR/m/libgtmutil.so /opt/worldvista/EHR/m(/opt/worldvista/EHR/m)

I am now getting a new error:

kdt0p@ubuntu64:~$ yottadb -run GDE

%GDE-I-LOADGD, Loading Global Directory file
        /opt/worldvista/EHR/g/mumps.gld

%GDE-E-INPINTEG, Input integrity error -- aborting load

[Sam Habiel]

I talked to Kevin on the phone, and he is moving from an x86 to an x64
system. GDE doesn't know how to do that upgrade; so I gave him
instructions on how to recreate the GDE file and to double check his
dse to make sure that he didn't mupip set some stuff over the years
that he forgot about.

--Sam

> To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/74c97d6e-7591-434c-aa4e-89a77b7bb4d3n%40googlegroups.com.

[Kevin Toppenberg]

I talked to Sam by phone, and it appears my problem is that my old VistA was 32bit, and GDE doesn't currently have the ability to change formats between 32 bit --> 64bit.

So these are the steps we took:

$gtm_dist/dse d -f &> /home/kdt0p/tmg_dse_export.txt
kdt0p@poweredge:~$ cat tmg_dse_export.txt

File      /opt/worldvista/EHR/g/mumps.dat
Region    DEFAULT


File            /opt/worldvista/EHR/g/mumps.dat
Region          DEFAULT
Date/Time       05-MAY-2021 16:55:43 [$H = 65869,60943]
  Access method                          BG  Global Buffers                7000
  Reserved Bytes                          0  Block size (in bytes)         4096
  Maximum record size                 32767  Starting VBN                   129
  Maximum key size                      510  Total blocks            0x003A19CB
  Null subscripts                     NEVER  Free blocks             0x000E6F74
  Standard Null Collation             FALSE  Free space              0x00006000
  Last Record Backup     0x0000000000000001  Extension Count              20000
  Last Database Backup   0x0000000004C8B6AA  Number of local maps          7437
  Last Bytestream Backup 0x0000000000000001  Lock space              0x00000028
  In critical section            0x00000000  Timers pending                   1
  Cache freeze id                0x00000000  Flush timer            00:00:01:00
  Freeze match                   0x00000000  Flush trigger                 6562
  Current transaction    0x000000030E6C7609  No. of writes/flush              7
  Maximum TN             0xFFFFFFFFDFFFFFFF  Certified for Upgrade to        V6
  Maximum TN Warn        0xFFFFFFFF5FFFFFFF  Desired DB Format               V6
  Master Bitmap Size                     64  Blocks to Upgrade       0x00000000
  Create in progress                  FALSE  Modified cache blocks            3
  Reference count                         4  Wait Disk                        0
  Journal State                          ON  Journal Before imaging        TRUE
  Journal Allocation                   2048  Journal Extension              100
  Journal Buffer Size                  2312  Journal Alignsize             4096
  Journal AutoSwitchLimit           8388548  Journal Epoch Interval         300
  Journal Yield Limit                     8  Journal Sync IO              FALSE
  Journal File: /opt/worldvista/EHR/g/mumps.mjl
  Mutex Hard Spin Count                 128  Mutex Sleep Spin Count         128
  Mutex Queue Slots                    1024  KILLs in progress                0
  Replication State                     OFF  Region Seqno    0x0000000000000001
  Zqgblmod Seqno         0x0000000000000000  Zqgblmod Trans  0x0000000000000000
  Endian Format                      LITTLE  Commit Wait Spin Count          16
  Database file encrypted             FALSE  Inst Freeze on Error         FALSE
  Spanning Node Absent                FALSE  Maximum Key Size Assured     FALSE
  Defer allocation                     TRUE

This is the characteristics of my existing, old mumps.dat.   This will be important if I had ever used mupip in the past to directly make a change to the database, which would then not be reflected in my .gld file.

Next I went into GDE and output a dump of my old GDE data.

kdt0p@poweredge:~$ mumps -run ^GDE

GDE> show -command -file=/home/kdt0p/GDE_dump.txt
GDE> exit

Which gives me a file containing the following.

TEMPLATE -REGION -COLLATION_DEFAULT=0
TEMPLATE -REGION -EPOCHTAPER
TEMPLATE -REGION -NOINST_FREEZE_ON_ERROR
TEMPLATE -REGION -JOURNAL=(ALLOCATION=200,AUTOSWITCHLIMIT=8388600,BEFORE_IMAGE,
TEMPLATE -REGION -KEY_SIZE=64
TEMPLATE -REGION -NULL_SUBSCRIPTS=NEVER
TEMPLATE -REGION -NOQDBRUNDOWN
TEMPLATE -REGION -RECORD_SIZE=256
TEMPLATE -REGION -NOSTDNULLCOLL
!
TEMPLATE -REGION -NOJOURNAL
!
TEMPLATE -SEGMENT -ACCESS_METHOD=BG
TEMPLATE -SEGMENT -ALLOCATION=100
TEMPLATE -SEGMENT -BLOCK_SIZE=1024
TEMPLATE -SEGMENT -DEFER_ALLOCATE
TEMPLATE -SEGMENT -NOENCRYPTION_FLAG
TEMPLATE -SEGMENT -EXTENSION_COUNT=100
TEMPLATE -SEGMENT -GLOBAL_BUFFER_COUNT=1024
TEMPLATE -SEGMENT -LOCK_SPACE=40
TEMPLATE -SEGMENT -MUTEX_SLOTS=1024
TEMPLATE -SEGMENT -RESERVED_BYTES=0
!
TEMPLATE -SEGMENT -ACCESS_METHOD=MM
TEMPLATE -SEGMENT -ALLOCATION=100  
TEMPLATE -SEGMENT -BLOCK_SIZE=1024
TEMPLATE -SEGMENT -DEFER
TEMPLATE -SEGMENT -DEFER_ALLOCATE
TEMPLATE -SEGMENT -NOENCRYPTION_FLAG
TEMPLATE -SEGMENT -EXTENSION_COUNT=100
TEMPLATE -SEGMENT -GLOBAL_BUFFER_COUNT=1024
TEMPLATE -SEGMENT -LOCK_SPACE=40
TEMPLATE -SEGMENT -MUTEX_SLOTS=1024
TEMPLATE -SEGMENT -RESERVED_BYTES=0
!
TEMPLATE -SEGMENT -ACCESS_METHOD=BG
!
DELETE -REGION DEFAULT
DELETE -SEGMENT DEFAULT
ADD -REGION DEFAULT -DYNAMIC_SEGMENT=DEFAULT
!
ADD -SEGMENT DEFAULT -FILE_NAME="/opt/worldvista/EHR/g/mumps.dat"
!
LOCKS -REGION=DEFAULT
!

<more to come> ...

[Kevin Toppenberg]

Next, I copied the GDE_dump.txt file to me new VistA 64 bit system.

I renamed my old 32 bit mumps.gld file --> mumps.gld.old

Next, following the guidance here: https://www.hardhats.org/projects/New/InstallVistAOnGTM.html, I typed

mumps -run ^GDE < ~/GDE_dump.txt |& tee ~/gde_setup_out.txt

This didn't seem to work, as per below. 

%GDE-I-GDUSEDEFS, Using defaults for Global Directory
        /opt/worldvista/EHR/g/mumps.gld

GDE>
GDE>
GDE>
GDE>
GDE>
%GDE-I-VALTOOSMALL, 200 is less than the minimum of 2048 for a ALLOCATION

%GDE-E-OBJNOTCHG, Not changing region template

GDE>
GDE>
-<snip>-
GDE>
GDE>
GDE>
%GDE-I-VERIFY, Verification OK

%GDE-I-GDCREATE, Creating Global Directory file
        /opt/worldvista/EHR/g/mumps.gld

So next I went into GDE and started adding the commands manually line-by-line via copy and paste. 

I realized that one line had been truncated:

TEMPLATE -REGION -JOURNAL=(ALLOCATION=200,AUTOSWITCHLIMIT=8388600,BEFORE_IMAGE,

should have been

TEMPLATE -REGION -JOURNAL=(ALLOCATION=200,AUTOSWITCHLIMIT=8388600,BEFORE_IMAGE,BUFFER_SIZE=2307,EXTENSION=100)

So I had to manually fix that.  (I don't know why that would have happened, but heads-up to others in the future!)

But even after fixing, I get another error:

GDE> TEMPLATE -REGION -JOURNAL=(ALLOCATION=200,AUTOSWITCHLIMIT=8388600,BEFORE_IMAGE,BUFFER_SIZE=2307,EXTENSION=100)
%GDE-I-VALTOOSMALL, 200 is less than the minimum of 2048 for a ALLOCATION

%GDE-E-OBJNOTCHG, Not changing region template

Looking into this error here: https://docs.yottadb.com/AdminOpsGuide/ydbjournal.html  I found this:

ALL[OCATION]=blocks

Sets the allocation size of the journal file. YottaDB uses this information to determine when it should first review the disk space available for the journal file. The size of the journal file at creation time is a constant (depending on the YottaDB version) but once the journal file reaches the size specified by ALLOCATION, every extension produces a check of free space available on the device used for the journal file.

YottaDB issues informational messages to the system log whenever the free space available is not much more than the extension size. YottaDB provides these extension checks as an operational aid for identifying before space runs out that a file system holding the journal file is low on space. When there is no more free space available on the file system holding a journal file, YottaDB shuts off journaling for the corresponding database file.

The default ALLOCATION value is 2048 blocks. The minimum value allowed is 2048. The maximum value is 8,388,607 (4GB-512 bytes, the maximum journal file size).

So I just changed 200 -> 2048
TEMPLATE -REGION -JOURNAL=(ALLOCATION=2048,AUTOSWITCHLIMIT=8388600,BEFORE_IMAGE,BUFFER_SIZE=2307,EXTENSION=100)

I was able to copy and paste the rest of the lines in without problems.

Then exited:

GDE> exit
%GDE-I-VERIFY, Verification OK

I then ran this command on my new database:

dse d -f
Region          DEFAULT
Date/Time       05-MAY-2021 17:47:56 [$H = 65869,64076]

  Access method                          BG  Global Buffers                7000
  Reserved Bytes                          0  Block size (in bytes)         4096
  Maximum record size                 32767  Starting VBN                   129
  Maximum key size                      510  Total blocks            0x003A19CB

  Null subscripts                     NEVER  Free blocks             0x000EA5B7

  Standard Null Collation             FALSE  Free space              0x00006000
  Last Record Backup     0x0000000000000001  Extension Count              20000
  Last Database Backup   0x0000000004C8B6AA  Number of local maps          7437
  Last Bytestream Backup 0x0000000000000001  Lock space              0x00000028

  In critical section            0x00000000  Timers pending                   0

  Cache freeze id                0x00000000  Flush timer            00:00:01:00
  Freeze match                   0x00000000  Flush trigger                 6562

  Freeze online                       FALSE  Freeze online autorelease    FALSE
  Current transaction    0x000000030BAF6C6D  No. of writes/flush              7

  Maximum TN             0xFFFFFFFFDFFFFFFF  Certified for Upgrade to        V6
  Maximum TN Warn        0xFFFFFFFF5FFFFFFF  Desired DB Format               V6
  Master Bitmap Size                     64  Blocks to Upgrade       0x00000000

  Create in progress                  FALSE  Modified cache blocks            0
  Reference count                         1  Wait Disk                        0
  Journal State               [inactive] ON  Journal Before imaging        TRUE

  Journal Allocation                   2048  Journal Extension              100
  Journal Buffer Size                  2312  Journal Alignsize             4096
  Journal AutoSwitchLimit           8388548  Journal Epoch Interval         300
  Journal Yield Limit                     8  Journal Sync IO              FALSE
  Journal File: /opt/worldvista/EHR/g/mumps.mjl
  Mutex Hard Spin Count                 128  Mutex Sleep Spin Count         128
  Mutex Queue Slots                    1024  KILLs in progress                0
  Replication State                     OFF  Region Seqno    0x0000000000000001
  Zqgblmod Seqno         0x0000000000000000  Zqgblmod Trans  0x0000000000000000
  Endian Format                      LITTLE  Commit Wait Spin Count          16
  Database file encrypted             FALSE  Inst Freeze on Error         FALSE
  Spanning Node Absent                FALSE  Maximum Key Size Assured     FALSE

  Defer allocation                     TRUE  Spin sleep time mask    0x00000000
  Async IO                              OFF  WIP queue cache blocks           0
  DB is auto-created                  FALSE  DB shares gvstats             TRUE
  LOCK shares DB critical section     FALSE  Read Only                      OFF
  Recover interrupted                 FALSE
  Reorg Sleep Nanoseconds                 0

The above in bold were different compared to the old data base.  The values in green are new, and were not present in the old database.

I suspect that this is all OK.

After this, I am able to launch yottadb and get into vistA via do ^XUP.

Still more configuration to do as per Sam's guide.

Thank you Sam for the help!

Kevin