Creating a VFS interface to D3

40 views
Skip to first unread message

patric...@yahoo.com

unread,
Sep 1, 2007, 3:19:31 PM9/1/07
to OpenQM
I am going to create a VFS interface to allow QM to directly access a
d3 database. My plan is to utilize D3's cdemo example and interface
directly into d3. I will have D3 and QM on the same machine.

My questions.

#1: Can I use the CCALL interface doug created to link over to the D3
executable?
#2: If yes, then where do I find the docs

It seems the only other option is to directly alter the QM code with
the d3 code which I know is undesired and why the CCALL interface was
created.

- Patrick

Tony G

unread,
Sep 1, 2007, 9:04:19 PM9/1/07
to Ope...@googlegroups.com
Rather than linking into a proprietary VFS I recommend writing a VFS for
any target environment using FUSE (http://fuse.sourceforge.net/). This
will then allow any product, including QM to interface into the target with
nothing more than common Open/Read/Write to the host OS.

A FUSE interface to QM will allow other systems to do the same with QM as
the target. This could completely eliminate the need for tape and
proprietary data transfer routines between systems. This also opens doors
for many other interfaces into these MV products - the possibilities for
this for freeware and commercial offerings are staggering.

There are many language bindings for the FUSE library. This means
developers have the option to write a VFS into MV environments in many
different languages including C, C++, Perl, and C# (Mono).

Not to get too complex but this could also mean we could create a QM BASIC
binding, which would allow developers to use QM BASIC to write file systems
which can then be used by other programs to do things in QM or other
environments. Again, the technical and commercial possibilities for this
are staggering.

I just haven't had the time to do this stuff, so I'm offering these ideas
for anyone here. Oh yes, and if someone wants to commission me to write
components as described here, I'd be quite happy to do so.

Tony Gravagno
Nebula Research and Development
TG@ remove.this.pleaseNebula-RnD.com

Doug Dumitru

unread,
Sep 3, 2007, 12:02:10 PM9/3/07
to Ope...@googlegroups.com
patr...@spm1.com wrote:
> I am going to create a VFS interface to allow QM to directly access a
> d3 database. My plan is to utilize D3's cdemo example and interface
> directly into d3. I will have D3 and QM on the same machine.
>
> My questions.
>
> #1: Can I use the CCALL interface doug created to link over to the D3
> executable?

I will dive in here.

CCALL should work, but linking in D3's C layer might have not expected
issues. You are also completely on your own, so keep the layer's simple
and well isolated.

> #2: If yes, then where do I find the docs

The CCALL layer is a low-level "stack" processors. It allows you to
build the native C calling stack and then "call" a function. It assumes
that you are very careful and understand calling convensions.

There are simple docs in 'op_ccall.c'.

Not sure if the attachment will get thru, but I have included a zip of a
file called CCALL.BP which has a bunch of examples, some simple, some
ugly. This is a "directory" file from a Linux system, so you might need
to massage it a bit to get the source onto windows.

If you do write to CCALL, be very careful. Double and triple check your
buffers to be sure you don't leave holes for overruns. Also watch out
for callbacks and signal handlers. These can be "tough" to handle.

> It seems the only other option is to directly alter the QM code with
> the d3 code which I know is undesired and why the CCALL interface was
> created.

Plus what you write can work on commercial QM (and I highly encourage
you to work on the commercial versions and contribute back what you
write where appropriate).

Other options you could consider would be an "over the wire" method.
With a simple TCP server on D3, you could export the file system with a
helper. We have done this before with Coyote RPC's over HTTP and it
works well, but there is a bit of setup and you might want to play with
something simpler. Also Coyote RPC is "out of process" and your
approach would have a 1:1 process relationship which makes handing some
structures like active select lists easier.

>
> - Patrick
>
>
> >

ccall.zip

patric...@yahoo.com

unread,
Sep 3, 2007, 2:51:25 PM9/3/07
to OpenQM
Unless I am wrong, it seems the VFS interface has been removed from
the latest open source versions (probably when it was moved to a
class). Therefore this discussion is now academic. Tony's suggestion
is now more interesting. I did some testing yesterday with fuse's
demonstration hello world example and it appears to do some style of
connection pooling. I will have my c guy play with this a little next
week and see what we can come up with.

The real issue with using fuse is no actual index/query support. It
seems to be although a good fit for reading/writing items. I utilize
a web call anyways when doing selects therefore this will not be much
of an issue. On the other hand, fuse appears to pool connections and
it would seem that you can have a persistant connection (kinda like
coyote).

Example
QUERY='SELECT LOANS WITH FUND.DATE = "9-10"'
QUERY<-1>='SELECT LOANS WITH REP = "ARGG"'
CALL QUERY.RTNE(REMOTE.HOST,QUERY,OPTIONS,RETURN.LIST,ERRORS); * think
sql returning a cursor
* NOW LOOP THRU RETURN LIST AND DIRECTLY READ ITEMS

* This process is better even with OSFI's in D3 that trying to query
across the OSFI. This allows stacked queries, optimization, security,
etc.

- Patrick

- Ptrick

On Sep 3, 9:02 am, Doug Dumitru <d...@easyco.com> wrote:

> ccall.zip
> 25KDownload- Hide quoted text -
>
> - Show quoted text -


GlenB

unread,
Sep 13, 2007, 4:54:19 PM9/13/07
to OpenQM

I started an RFC many years ago relating to this kind of
communications issue. It's just a guideline to storing and retrieving
data from an MV data source, using an ASCII protocol over any kind of
media. Maybe this can give you some alternative ideas. Like all the
other things I don't have time to do, I hope to finish it one day.

http://mvdevcentral.com/MV-RFC.txt

I wrote a D3 socket service using this partial RFC and also a VB
application that let me pull up data remotely. The VB app may be gone,
but I think I still have the sample service. Since it's plain-text, I
used telnet to test things. You could use any language you wanted
provided you can parse text and access the service.

Glen

> > - Show quoted text -- Hide quoted text -

Reply all
Reply to author
Forward
0 new messages