YottaDB upgrade notes

[Kevin Toppenberg]

Not a question.  I started a note asking for help, but figured it out.  Will post for future reference.

I am upgrading from YottaDB r1.32 to r2.02.

Details:

• I have compiled yottadb from source.  After successful compilation, I run ./ydbinstall --overwrite-existing.  This puts all the files into /usr/local/lib/yottadb/r202/
• I shut down my vista cleanly via: sudo systemctl stop vista.  I confirmed that it was not running via systemctl list-unit-files --type=service | grep vista which showed that it was enabled, but not running.
  
• My vista startup scripts reference the yottadb files via /opt/yottadb, which was a softlink to /usr/local/lib/yottadb/r130.  I renamed that link to yottadb0 and made a new soft link such that /opt/yottadb -->  /usr/local/lib/yottadb/r202/
• I reviewed the release notes here: https://gitlab.com/YottaDB/DB/YDB/-/releases, and paid particular attention to the sections: UPGRADING TO YOTTADB r_______
• I copied my prior journal files to a folder named "kill_later"
• I restarted vista cleanly via sudo systemctl start vista
• I thought this might be sufficient, so I attempted to run CPRS, our GUI that connects to yottadb vista server.  But I got a connection error, which usually means server error.
• I launched yottadb to the direct mode console and attempted to run ^XTER.  I then got an error about errors in $ZROUTINES, esp related to
  • $gtm_dist/plugin/o/_ydbmwebserver.so
    
  • $gtm_dist/plugin/o/_ydbposix.so 
  • $gtm_dist/plugin/o/_ydbgui.so
• I looked in the file system, and these files are not present, so I fixed by copying the plugin directory from the prior yottadb folder to the new one.
  
• I stopped and restarted vista via sudo systemctl start vista.
• I am not getting the error:
  • %YDB-E-GDINVALID, Unrecognized Global Directory file format: /opt/worldvista/EHR/g/mumps.gld, expected label: GTCGBDUNX115, found: GTCGBDUNX112     At M source location XTER+4^XTER
• I fixed this by running gde from the linux prompt, then exiting.  It stated that it was updating the file.  I was then able to run vista code without the above error. 
  

NOTE: After switching to the codebase where I had made changes, I found a bug.  So I tried to switch back to r132.  But then it couldn't handle the new global directory file.  So I then tried to download a clean r202 from yottadb.  It wouldn't install on my Ubuntu 2020 LTS.  So I ended up having to upgrade frm 2020 --> 2022.  I then also upgraded 2022-->2024.  I was then able to get clear r202 and I'm back up and running.  What a lot of work!  I wish I had backed up the global directory.    My entire server is a virtual machine that we back up every night.  I could have restored that, but that would have taken about the same length of time, so I chose the first path.

[K.S. Bhaskar]

Kevin –

Since you didn't ask questions, I will! Of course, Bhaskar would not be Bhaskar without a couple of gratuitous comments thrown in…

• Are you using --overwrite-existing because you have your own custom prior builds? To keep myself from getting confused, I use directories lik …/r200 and …/r202 for production releases, and directories such as …/r201 and …/r203 for builds other than production releases.
• lsof would seem to be a safer way to ensure a database has no processes accessing it. While you may have VistA installed as a service, does that really ensure that no YottaDB processes have a database open?
• Before copying your prior version journal files to another directory, doing a MUPIP SET JOURNAL with the new YottaDB build to switch journal files would be preferable. Otherwise, the database file header may have a pointer to a journal file that doesn't exist.
• Since plugins with C code are compiled with header files that are specific to a YottaDB release, using plugins compiled with one release of YottaDB with another release of YottaDB is not safe. Similarly, since object code can change, you should not use M object code compiled with one release of YottaDB in another. We protect against this when going from one production release to another, e.g., r2.00 to r2.02, but not against builds of code in flight between releases.
• Yes, the global directory format can change at any time. YottaDB will upgrade global directories, but not downgrade them. So we recommend backing up global directories before switching a database to a new release, or at least doing a SHOW COMMAND with the old release, and saving the output.

Thank you for posting your comments.

Regards

– Bhaskar

[Kevin Toppenberg]

On Monday, January 6, 2025 at 3:15:48 PM UTC-5 K.S. Bhaskar wrote:

Kevin –

Since you didn't ask questions, I will! Of course, Bhaskar would not be Bhaskar without a couple of gratuitous comments thrown in…

• Are you using --overwrite-existing because you have your own custom prior builds? To keep myself from getting confused, I use directories lik …/r200 and …/r202 for production releases, and directories such as …/r201 and …/r203 for builds other than production releases.

Yes, it had my prior custom build in it.  I like your directory naming idea. 

 

• lsof would seem to be a safer way to ensure a database has no processes accessing it. While you may have VistA installed as a service, does that really ensure that no YottaDB processes have a database open?

I saw those instructions in the upgrade guides, and I did it at one point.  But I was thrashing around and turning it on and off while I was trying to confirm that I had a bug.  So I should have done better and will be more careful next time. 

 

• Before copying your prior version journal files to another directory, doing a MUPIP SET JOURNAL with the new YottaDB build to switch journal files would be preferable. Otherwise, the database file header may have a pointer to a journal file that doesn't exist.

I didn't know this.  Thank you.

 

• Since plugins with C code are compiled with header files that are specific to a YottaDB release, using plugins compiled with one release of YottaDB with another release of YottaDB is not safe. Similarly, since object code can change, you should not use M object code compiled with one release of YottaDB in another. We protect against this when going from one production release to another, e.g., r2.00 to r2.02, but not against builds of code in flight between releases.

This is my plugin dirctory.

:/opt/worldvista/EHR/m/plugin/o$ ls -Al
total 876
dr-xr-xr-x 2 root root   4096 Jan  5 22:37 utf8
-r--r--r-- 1 root root 462120 Jan  5 18:49 _ydbgui.so
-r--r--r-- 1 root root 285960 Jan  5 18:49 _ydbmwebserver.so
-r--r--r-- 1 root root 142376 Jan  5 18:49 _ydbposix.so

does the ydbinstall program provides these plugins?  I think it was when I was using my custom build that I was missing the plugins.  I think when I did a clean install I told it to install all the plugings and wrappers.  So I think these one should be good.  (I hope?)  Not sure how to tell. 

 

• Yes, the global directory format can change at any time. YottaDB will upgrade global directories, but not downgrade them. So we recommend backing up global directories before switching a database to a new release, or at least doing a SHOW COMMAND with the old release, and saving the output.

I was able to read the manual and get a show command listing.  But with scanning I could not figure out how to use this to remake a global directory in the older format.    If gde created a backup of the file whenever it makes a change, that might be nice.

Thanks for your help.