INDEX.CLS?

[Gary]

Group:

I read something on the net about an "INDEX.CLS" that simplifies
access to files via alternate keys. I would like to incorporate this
functionality in DICTFILE.CLS. Does anyone know where I can find the
source to INDEX.CLS (it doesn't seem to be in the svn...

Thanks,

GaryW

[eppick77]

In the commercial version the program is stored in the BP file in
QMSYS.

I am not sure if it is available for the GPL version. Martin wrote it
when I asked for that functionality. I am using it and it is really
nice to be able to traverse the index one record at a time rather than
one key at a time.

It returns to you the ID as well as the data that was put into the
index.

Eugene

[Martin Phillips]

Hi Gary,

This is released only as part of the commercial product. Copying it to an
open source project would be a contravention of the licence conditions.

Martin Phillips
Ladybridge Systems Ltd
17b Coldstream Lane, Hardingstone, Northampton, NN4 6DB
+44-(0)1604-709200

[Diccon]

Gary. It seems the feature's source is not in the GPL version.
If you, me or someone else wanted to write a piece of functionality
like that into GPL qm/Scarlet then that's fine.
I will chuck it on the ever growing wish list. We can discuss what it
would require and I can point you towards the structures code that
will help.
I will try and be on the IRC channel tonight (UK time) between 19:00
and 01:00 am.

Until Intellectual Property Rights enforce copyright on ideas and
designs of software there is nothing to stop this (IPR is a LONG was
off from being accepted).
What Martin is describing is the terms if use of the commercial
software licence. So you wouldn't be able to use the commercial
version.

Also, actually having trouble repeating the bugged Sort/select query
on my personal dev system. The one with long delays. Since you've
caused the issue on a vanilla system I think I might need your help in
re-creating the circumstance. I've got a circumstance where I can
cause it on a running system, but it's obviously not the data as even
copying that doesn't trigger it. REALLY want to hunt this one down and
fix it.

-Diccon

On 17 Aug, 08:44, "Martin Phillips" <martinphill...@ladybridge.com>
wrote:

[Gary]

Diccon,

Thanks for your reply. Well, if INDEX.CLS is part of the
commercial version, I'll just have to write equivalent code for the
open source version. It's not a big deal, it will just require a
little more time.

It's should be easy to reproduce the sort problem. It should also
be easy to fix. The problem is that sort items are inserted into a
binary tree with no tree balancing at all. I REALLY believe that
maintaining a balanced tree will fix the problem. To reproduce the
problem, make a reasonably large file with identical keys. Sort only
on that key. The bad sort performance would also occur if all the keys
were already in order. However, due to the hashing done in OpenQM, it
is necessary for all the items to be retrieved in key order to be
inserted into the sort. The safest way to preproduce this is simply
use identical keys. I'd be happy to send you my file. :-) It has
65000 records. Each record has about 32 characters. The sort key is
4 characters long and (right now) all the keys are the same ("0001").

GaryW

[Ashley Chapman]

> It's should be easy to reproduce the sort problem. It should also
> be easy to fix. The problem is that sort items are inserted into a
> binary tree with no tree balancing at all. I REALLY believe that
> maintaining a balanced tree will fix the problem. To reproduce the
> problem, make a reasonably large file with identical keys. Sort only
> on that key. The bad sort performance would also occur if all the keys
> were already in order. However, due to the hashing done in OpenQM, it
> is necessary for all the items to be retrieved in key order to be
> inserted into the sort. The safest way to preproduce this is simply
> use identical keys. I'd be happy to send you my file. :-)

That would be a good idea. Perhaps it can go onto the Crimson site as
a reference file. It might even encourage me to finish off the
benchmark tool I've been meaning to complete. :-)

Ashley

[Diccon]

Zip it and ship it by all means. (Address above)
So we think it's an un-balanced B-Tree, Ok. Think I can work with
that. Bemused that algorithms that don't balance still exist. Had that
hammered into my head when I was taught. If it isn't balanced
dynamically as it builds then it certainly won't continue to build
efficiently.

I Have literally just found the missing link for the way of finding
relevant code for operations like SSELECT. If your not ever going to
try and backtrace the logic of what is being executed you probably
don't want to read this :)

BCOMP (Basic) lays the op code in the binary. Which is picked up when
run by the virtual machine (C code) and runs the op code subroutine
(usually in op_dio or op_something, again C code) which then can call
k_recurse(), which pulls up the compiled internal BASIC subroutines
(e.g. _SSELECT) ...Most of which call little functions which are
actually written in C....confused yet? You should be.
I'll find a spot on the wiki for the above and flesh it out.

Just glad I can finally see what is calling what.
The B-Tree is created by repeated calls to op_sort.c/sortadd() in
GPL.BP/_SSELECT
It's moments like this I wish I could close my eyes, black box test
it, and say "It's fine, honest" lol
Going to have to do a pencil and paper run through of the BTree build,
me thinks :P

-Diccon

[Tom Potts]

2009/8/18 Diccon <diccon...@gmail.com>

Zip it and ship it by all means. (Address above)
So we think it's an un-balanced B-Tree, Ok. Think I can work with
that. Bemused that algorithms that don't balance still exist. Had that
hammered into my head when I was taught. If it isn't balanced
dynamically as it builds then it certainly won't continue to build
efficiently.

I Have literally just found the missing link for the way of finding
relevant code for operations like SSELECT. If your not ever going to
try and backtrace the logic of what is being executed you probably
don't want to read this :)

BCOMP (Basic) lays the op code in the binary. Which is picked up when
run by the virtual machine (C code) and runs the op code subroutine
(usually in op_dio or op_something, again C code) which then can call
k_recurse(), which pulls up the compiled internal BASIC subroutines
(e.g. _SSELECT) ...Most of which call little functions which are
actually written in C....confused yet? You should be.

I only wish I was; I had to figure it out once before.  Unfortunately I could never explain to anyone else the process.  :o)
 

I'll find a spot on the wiki for the above and flesh it out.

Good idea, but I'm not sure it's actually possible to explain.

"No-one can be told what the Matrix is... you have to see it for yourself."

Tom
 

[Gary]

Diccon,

Let me know when you start digging into the sort so that we don't
duplicate effort. It was my intent to fix the sort, but I probably
won't get to it for a couple of weeks (need to finish a project that
MIGHT actually make me money first). If you get started on it, I will
move to something else. I don't think it will be a tough fix. I'd be
surprised if I can't fix it in a day. The algorithms are well known
and the existing code is fairly easy to follow.

GaryW


On Aug 18, 9:17 am, Tom Potts <karake...@gmail.com> wrote:
> 2009/8/18 Diccon <diccon.tes...@gmail.com>

[Diccon]

Will do, away to Germany until next Tuesday anyway and have a busy
week following that. But I will let you know before I make any changes
so we don't tread on each other's toes. Thanks.
-Diccon