Moving data from fileman/VistA to SQL

[Abdullah Bin Zahid]

Hi,

 

We are doing a clinical research and for that we need convert data in Patient, V POV, PTF, prescription, pharmacy patient and imaging files to SQL and anonymize during the process, and create a log file to keep track of things. So I initially tried "Data export to foreign format" for each file, hoping that I will be able to import CSV into SQL. But facing problems, like very big .txt files, and difficulty getting approval to have software needed (like PHP, mySQL, SQL,) on VA computers, Query takes too long, at times gets interrupted etc.

 

Any ideas on how may I solve this issue? Can I directly convert data from these files into SQL/ Access database, without having to go through CSV and anonymize?

[Kevin Toppenberg]

In the VA, isn't there already an SQL interface, "MDWS" ("meadows") or something like that?

Kevin

[Ab]

I just started using Fileman. Most of my Fileman knowledge is based on Fileman Manuals.
So, Kindly can you please tell me from where I can get more information about "MWDS" and where it is located in fileman options hierarchy?

[David Whitten]

MDWS is NOT an SQL Interface. It is C# code that calls the RPC interface for a VistA instance, and the SQL interface for non-VistA warehoused data.

Running an SQL engine to deal with the VistA data is has problems because of the limited nature of SQL. An MS Access database would

have even more problems, .

I would think hard before not anonymizing the data. I expect your Research Board will have that as a requirement to allow you to even look at the VA data.

Perhaps Conor Dowling can help you with querying using FMQL, as it is a variant of SPARQL. Even that will require adding software to the VA machines to extract the projection of the VA data.  Alternately, KB-SQL or Cache SQL would give you access to VA data, but again will require adding software to the VA machines, or adding data definitions for the Cache SQL.

Perhaps you could get permission to use a backup of the VistA system and then add the software you need onto a machine running the backup. I expect you will find more success using the native tools in VistA to manipulate the data rather than using tools that are not well suited, but I don't know your budget nor your programming resources.

--
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+u...@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "Hardhats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sergey Gavrilov]

There is a tool called M-Data Extractor (MDE), developed by the Strategic Reporting Systems, Inc. This is a client/server application that allows to define rules for data extraction and load the data from FileMan files directly to a SQL server (MS or Oracle).

This tool was (and maybe still is) used in the VA. For example, it was used by the Clinical Case Registries group from Palo Alto, CA to extract Hepatitis C and HIV related data from VistA.

Best regards,
Sergey Gavrilov

P.S. I am in no way related to the aforementioned company.

[Ab]

Thank you so much Kevin, David and Sergey.
I was able to do this as follows.
Use data export in fileman to export all required data in the form of CSV, then import these huge text files (500 MB to 2 GB) in SPSS statistics, and then export from there to Access.
I had to take this long detour, because SPSS can work with 250+ columns, and millions of rows.

[Andy Bruce]

http://themorningconsult.com/2014/06/switzerland-approach-might-become-key-part-wearable-health-platforms/

Am I the only one who hasn't heard of the EPIC/Apple agreement announcement?

"The two most interesting aspects of the Apple announcement were the
endorsement from Mayo Clinic and an undefined partnership with Epic. On
the surface, Apple and Epic don’t have a lot in common with very
different underlying technologies, so a shared developer ecosystem is
hard to imagine.

This brings up as many questions as challenges...

I hope the community is paying attention to the commoditization of
service components based on interoperability and aggregation. These are
capabilities the big players appear to be interested in.

Food for thought...Do we plan to let this new wave of divvying up
marketshare pass us by because we cannot organize effectively enough to
be taken seriously?