Version Control

[Sérgio Pereira]

Hi all,

I'm brand new to the Pick/ MultiValue system.

I'd like to know if anyone has ever implemented a solution to support file version control outside of the PICK system.

Our goal is to enable Continuous Integration/ Continuous Delivery around a PICK (Reality) system.

I am aware that the PICK system we're using (Reality) has built-in source control capabilities, but we need to take it one step further and be able to perform the following from a 3rd party version control system (most likely Git):

- database + BASIC code backup

- restore data files within the PICK system to a previous version

- inline diff file compares

So roughly this is my proposed approach:

- have a remote PICK system to be used as a development environment. Each developer in the team would have their own dev database within the shared environment.

- have an external (maybe UNIX based, potentially the current developer's machine) system where all the PICK data files, BASIC code etc are stored. This would be a snapshot of all the current data files and file structure in the PICK system.

- each developer would have a single local version control repository; a remote branch would have all the latest changes pushed by all the developers (central repository).

- rely on the external host to act as a local repository and perform various version control activities: local repository, commit, push, version comparison, merge, checkout, rollback, etc.

- retrieve latest data files from PICK system/ use file transfer tools to copy files over to the source control local repository (remote computer). This would occur at least once per day and whenever the developer is happy with their latest changes.

- send files to PICK system from the aforementioned local version control (user) host. This would override the files in the PICK system and would be performed every day by each developer. This would ensure the latest changes performed by the team are available in the developer's test database.

- create a package with the latest code changes and deploy it onto a PICK host system

Any guidance would be much appreciated.

Thanks,

Sergio

[Tony Gravagno]

Sergio - WELCOME!! :)

With full respect, I think you'll need to partner with someone one this.
I work with FOSS and business projects on GitHub, and with local and remote SVN repos. So I know what you want to do.

But in the Pick / MV system there is more than just code. If you want to do real version control, you'll need to track changes to M/DICT items (verbs=commands). You'll also need to track file dictionary definitions, which is the same as any other database schema where you might need to track changes made with ALTER TABLE.

When you're talking about restoring data, it sounds like you mean user data. Putting an entire database into a VCS repo is unusual and not practical. There are other mechanisms for doing this. There is a backup process called a FILE-SAVE, where the media can be used to selectively restore desired data.

The fast technical answer to what you're doing is that everything needs to be exported to/from the DBMS hashed files so that it can be versioned. So you'll need to rely on triggers and code to do that marshalling of data. Diff is easy with external tools, especially since the versioned data needs to be outside.

To execute Diff, for example, at your Reality command prompt, move local data out, then "shell out" to invoke something like WinDiff against the local file and the repo. On return clean up the work files. That's the pattern for all of this.

Does this kind of code exist? There is a product called PRC but I don't think it's been ported to Reality, and I think it might be "too Pick" for your comfort level and expertise. I have my own code which I'd be willing to FOSS but it's not ready for that kind of effort yet.

There is also some FOSS that was written for another platform but should be close to what you want:
http://pickwiki.com/index.php/SVN
I know the company that published that and they are Intensely VCS-oriented, so I'd trust the code is good once ported.

So, what you want is just not built-in and perhaps not immediately available, but it's certainly possible and not too far away. Others here might offer their own home-grown solution - and I'd be happy to collaborate on anything that's offered as FOSS.

About partnering with someone on this - I strongly suggest you check with your Reality application vendor, Reallty support provider, and/or Northgate IS.

HTH
T

[Mario Bajada]

Hi T,
I have been considering something along the same lines, so checked your link to PickWiki
My question may be slightly OT but I shall accept the consequences and proceed.
The SVN wrapper as well other source on site use double square brackets [[ & ]]
Are you aware of their purpose and could you please elucidate on their denotation/usage?
RGrds M...

http://pickwiki.com/index.php/SVN

[Susan Joslyn]

Hi Everyone,

PRC is a 25 years mature, Pick-Multivalue-specific, completely software development life-cycle tool. It supports the model that you describe as well as other, more "MV-centric" models.  I would be happy to talk about it / email about it / give a web demo to anyone grappling with these ideas!

Some info at sjplus.com

sjo...@sjplus.com

[Tony Gravagno]

Mario - I saw that too and was puzzled by the syntax. Maybe one of our U2 colleagues can help us understand that.
I can ping Jim Adrig if he doesn't see this and ask.
That could be something internal at DocMagic for variable/subroutine tracing throughout their application. We lack the ability to do this in MV, aside from the AWESOME mvToolbox, and I wouldn't be surprised if DocMagic uses an editor that can trace/follow these references, with pre-processing to remove the syntax from their code just prior to compilation.

Unless someone tells us otherwise, I'd guess the action there would be to just remove all occurrences of the unique strings [[ and ]].

HTH
T

[Tony Gravagno]

Susan, whenever the topic of version control or regulatory compliance comes up, I always refer people to you and PRC. Could you tell us which platforms PRC supports right now?

One of my D3 clients is getting heavy into SOX compliance. Their management will ask for specific changes to respond to audits. But this is not my field of expertise, and while I can code whatever is required, I don't have insight into what details might be required in an implementation to be truly compliant. My current approach will be to ask for audit details and the relevant references to specific regulations, and then ensure my code complies. But if PRC already includes functionality that goes to this level of detail, I'd prefer not to repeat the effort. So with reference to the OP and this thread and version control, I'm hoping you can tell us more about how PRC ensures regulatory compliance, specifically with version control, or whatever else it also does.

Thanks.
T
(BTW, I miss your Dad.)

[James A]

If you look at the history of that page, you'll see that at some point someone ran a 'conversion script' of some kind that added all the [[]]'s.

So we should revert that or I could upload it again? (or just remove all the brackets...).

- James

On Wednesday, August 9, 2017 at 4:31:59 AM UTC-7, Mario Bajada wrote:

[Tony Gravagno]

Ahh, that makes sense, James. (Good to see ya!) The bracketing was obviously intended to serve as a base for clickable links to create new wiki pages. The conversion program they ran on it wasn't coded to avoid parsing within code blocks. As just another community guy here, I'd request that you re-post your latest code. Someone might offer to translate it for platforms that don't support the DEFFUN syntax (yet).

With the absolute best possible intentions Zumasys had for adopting the wiki, in actual implementation of it was not done well at all, and as we see here was only partially "done". But no one there was charged with fixing what was broken in the conversion process, or doing any kind of high-level maintenance afterward. So what we have left is this weird upgrade/downgrade from before the adoption - and I don't think anyone really uses it anymore. One of these days it will either be transferred again to give it another chance, given resources to keep it up properly, or retired for lack of interest. That last choice would be a real shame. I know that's not what was intended.

T

[Mario Bajada]

Thanks Tony, I too thought along the lines of some form of parsing or per-processing, though my blinkers were looking for syntax variations.

Thanks James, it took me a while to realise that by 'history' it wasn't those comments within the source but rather the history tab of the wiki.

[Susan Joslyn]

Hi T,

PRC is available on Universe and Unidata as a mature product. It is available in a beta state for Design Bais and is on it's way to being available on MVON# and with MVON's NetBuilder product, could also be available on D3.  Thanks for asking!

Yes, PRC will do what you need for the Sox audit, and we (PRC and I) know what they're going to ask for, too.

PRC provides both preventive and detective controls. This means you can set up users (or classes of users) that can't make any changes (preventive) or that are audited (detective). So that's a big chunk of what most of the regulator compliance guidance wants. PRC also controls who can deliver software from place to place... for example, perhaps the programmers can deliver their own work to a test environment. Or perhaps they can only do that if another sign-off is on there, too. But then maybe they can't deliver their own (or any) software to LIVE.

Talk to me about what your software is, maybe this is a chance for us to work together!

Susan

and p.s. thank you, I miss him, too.

[Tony Gravagno]

Thanks for the info. For anything specific I'll open a new thread. I think we have enough on-topic info in this thread.
Best,
T