Media Metadata

[Tony Gravagno]

George's thread about tape media reminded me of a persistent problem that I have.

Embarrassing confession time...

I don't catalogue what data is in all of my repositories. I don't record where code or data is. The name FILESAVE_150123.D3P doesn't tell me much, other than it's a D3 backup of whatever files were not DX'd from 23-Jan-2015. I don't know which system it came from, the D3 version, which accounts are there, or even why I have it.

I believe the data is required in the filename, because the timestamp on files gets changed as you move it around.

I do try to name backup files for their purpose, like NEBULAMAIL-3.1.1.D3P. But is that the version that still had POP3 and IMAP, or is that one SMTP-only? Is CLIENT_FOO-ACCOUNT-SAVE_BAR.vtf from before or after I made their mods? I need to go back to my time logs, emails, activity journals, or maybe Skype or Slack chats - to see if I referenced the filenames anywhere.

Sometimes, rarely, I put a media file into a unique folder, along with a ReadMe.txt that provides a clue about its purpose. THAT is MetaData.

In hindsight, an actual database for all of this metadata would have been really helpful to me over the years. It's not too late to create one. It IS too late to try to catalog most of the files I already have. Such a database would need to have attributes for details like:

- What accounts are in the filesave?

- Why was this created? What is it intended to be a snapshot of, and why is this point in time important?

- Is this simply a backup in case a system crashes?

- Was it sent to anyone?

- Was it used for product distribution?

- Was it a part of a project?

When we get a full backup, we need to know the projects or files that are of interest in the accounts. I've written a lot of code over the years. Some of these pet projects were saved as "cool but no one will buy it so I need to shelve it for now". Then they were removed from my live development system. Where are they now? Where is that cool routine that I wrote 15 years ago ... that I've probably re-written a few times?

I have that issue in my MV databases that are on my development systems right now. Which account has that one piece of code? For me, it might be NEBULAMASTER, NEBULA-INTERNAL, or in one of many special-purpose accounts like NEBULA.WS (an app that I wrote that turns BASIC subroutines into web service end-points). I've written routines for encryption, XML and JSON parsing, extracting firstname/lastname values from text, communications with Skype and Slack and other services, telephony and SMS, and so many other areas discussed in these forums. Was some of that lost in TEMP.BP, or in the WORK1 account that I deleted out of the VME?

Yeah, I confess - when I need some of this stuff that I sometimes use SEARCH-SYSTEM or FIND or whatever, and scan the entire database for text that might be included in the code that I'm trying not to re-write. And then I need to wonder ... Did I write that in D3? QM? Universe? On some client system?

And for communications, where there is a remote component, I rarely copy my MV code into the OS project (.NET, Java, etc) to archive it all as a single, sync'd client/server snapshot. I only do that for client/customer projects. So when I save the MV side into an archive 'here', and then save the "other stuff" 'there', how do I relate them? Well, I don't, and I search.

This isn't just about Pick stuff. I have documents and programs and other downloads in folders - zipped, then aggregated into tar files and sent off to cloud storage. It's a trip through Nostaligia land when I open one of these old archives.

And this isn't just about my stuff. Every company I know - yes, "Every" - saves files into archives that are later forgotten, and often lost. "What happened to the code for Foo Company?" "It might have been on Jane's system but we reformatted that after she left."

After all of that, to be clear, I treat client data with reverence, and no client data is ever lost. But it is archived after delivery and at some point I do lose track of which project is related to some timestamped archive.

So ...

- Do you use some system to track assets like this?

- If you have an asset tracking system does it include the kind of detail and attributes that I've noted here?

- Do you still lose stuff?

- For developers - do you ever use MV to track stuff like this?

- Does anyone use a more general purpose asset tracking system?

- Does anyone "tag" their data, like with a serial number and then track its location like inventory?

- Will anyone else here confess to similar pain?

After taking the time to do this open confession/catharsis, I'm sure I'll write something to address the problem. Right now I'm thinking about an app with a central server component, probably QM with a single-user Personal license in a cloud server. I'll create a Windows client, mobile app client, Slack client, and shell access from any Linux or Windows system. Then, whenever I save something somewhere, in my systems, a client's, cloud or elsewhere, I'll use one of these clients to create a tracking ID and note in QM, and I will (should) then leave the ID somewhere near the archive. To avoid having to use MV, maybe I'll use a Git repo (private, not GitHUB) and treat all such operations as a Commit, then store the Commit ID with the file, or as a part of the filename.

What would I Commit to? I dunno yet, maybe every metadata object becomes a new file in some organized file system? That's why I was thinking about a database.

Why QM? The Personal license is free for this personal application. I'm not going to pay a thousand dollars to put up a trivial D3 database like this. jBase and Universe would be good for this too but that's not where my comfort is. I'd love to use MVON Express but it's not a Linux product. I can use MySQL (MariaDB) for free too, even front-end this as a WordPress application, but I'd prefer to profile MV where possible. These are questions we alll face when trying to deploy any pet project like this.

There are many ways to skin this cat, and the tools may change. I have all of the tools and skills to do this stuff ... in some archives, folders, or accounts ... somewhere. I just need to decide what tools to use and then commit to doing it. But every mechanism will require discipline to create metadata as an ongoing, mandatory step in any archive process. I have the discipline if I have the right tools.

I'm just wondering how others deal with this - whether by process or by enduring the pain of not having a process.

And ... any end-users interested in sponsoring a new project? :) Gotta ask!

T

[euobeto .]

For each project I create a file and create 3 files inside it, $ log $ patch and regg for specific equates, and I use a svn with tracker, now I am changing my compiler the idea is: when compiling a program it will ask for the number  of the tracker ticket with this writes the $ patch to tracker information and commits it to Svn in case of compilation without errors

And I have I lightly menu with some fix routines, For each project I create a file and create 3 files inside it, $ log $ patch and regg for specific equates, and I use a svn with tracker, now I am changing my compiler the idea is: when compiling a program it will ask for the number  of the tracker ticket with this writes the $ patch to tracker information and commits it to Svn in case of compilation without errors

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com
To unsubscribe, email to: mvdbms+un...@googlegroups.com
For more options, visit http://groups.google.com/group/mvdbms
---
You received this message because you are subscribed to the Google Groups "Pick and MultiValue Databases" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mvdbms+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mvdbms/d408efbd-e8c1-4e99-8978-950c5acc17ab%40googlegroups.com.

--

Enviado do Gmail para celular

[Ian McGowan]

I agree with this point quite a bit - you can get pretty far with just a set of folders and a README.TXT as Tony suggests.  I have three use cases where this works pretty well (at least for a solo consultant - if working on a team it gets more complicated):

1) I use dropbox to keep stuff about clients and projects organized.  CLIENTS\BigBank\Projects\2019_04_FoobarConversion CLIENTS\BigBank\Invoices etc.  I have the same naming convention across all clients, and some other folders for stuff about my own company with a similar structure.  I usually name subfolders YYYY_MM_something, and that seems to work pretty well.

2) For technical notes, I've tried a bunch of different things, the last was Microsoft OneNote.  After some annoying sync failures, I have a private github repo called NOTES and keep everything as plain text/markdown files in that folder.  I can get to it from any PC with a web browser, or an app on my phone/ipad, and it's also stored in dropbox.

3) For code, there's obviously separate repos per client.  I always edit code locally on my PC and then sync it automatically to whatever system it belongs on (for testing, deployment to prod varies by client).  I've been getting away from having everything in one big BP folder, and try now to have a BP per project (e.g FOOBAR.BP).  This is also stored in dropbox.

I don't have any long term databases/accounts to try and track - for development purposes they are basically disposable for me, and it seems difficult to treat accounts as things of value - extract the code/dictionaries/docs into project files just like you would for the client side of things. For Tony's pain point, I'd extract all the compressed files into a new folder (maybe on a new drive), call it BIG_BALL_OF_MUD, and then every time something useful turns up from there, put it in it's proper place in the organized area.  Can't recommend enough two ideas: 1) Use dropbox/onedrive/whatever so every machine you use on a regular basis has all your stuff in one place, and 2) use git/fossil/svn to version every plain text file you work with - a lot of metadata needs revolve around knowing who changed something, when, and why..  Revision control addresses that...

To unsubscribe, email to: mvd...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms
---
You received this message because you are subscribed to the Google Groups "Pick and MultiValue Databases" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mvd...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mvdbms/d408efbd-e8c1-4e99-8978-950c5acc17ab%40googlegroups.com.