VMS Python IndexedFile - NotImplementedError: IndexedFile.primary_keynum

[Arne Vajhøj]

JFP VMS Python 2.7.18 on VMS Alpha.

Has anyone gotten vms.rms.IndexedFile working?

I am stuck at:
NotImplementedError: IndexedFile.primary_keynum

And a quick look at the IndexedFile __init__ and
primary_keynum make it look difficult to avoid.

Arne

[Jan-Erik Söderholm]

I have used vms.rms.IndexedFile for many years in routines
that reads RMS indexed files and loads Rdb tables.

But primarily for sequential access, not using key lookup.

I can see that I have this lines, not sure what they does,
probably just added from some RMS example from the
(now closed) JPF PythonVMS web page.

def primary_keynum(self):
return 0

The rest of the code never calls primary_keynum() and I have
no idea what happens if that is removed…

Jan-Erik.

[Arne Vajhøj]

So you create a class that inherits from IndexedFile and overrides
primary_keynum?

I will have to try that.

Arne

[Jan-Erik Söderholm]

If you say so, I guess I do... :-)
As I wrote, I just used one of JFP's examples.

>
> I will have to try that.
>
> Arne
>

I can mail you my complete script if you like.
Reads an RMS file and writes to a Rdb table.

[Jan-Erik Söderholm]

He he, just saw your email. I'll reply to that...

[Simon Clubley]

On 2022-08-28, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>
> So you create a class that inherits from IndexedFile and overrides
> primary_keynum?
>
> I will have to try that.
>

For the benefit of anyone searching for this in the future, what
was the solution ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

[Arne Vajhøj]

On 8/30/2022 1:40 PM, Simon Clubley wrote:
> On 2022-08-28, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> So you create a class that inherits from IndexedFile and overrides
>> primary_keynum?
>>
>> I will have to try that.
>
> For the benefit of anyone searching for this in the future, what
> was the solution ?

That solved the problem in the subject line.

But then I had another problem. Jan-Erik emailed me a
working example (thanks) and based on that I got it working.

Demo code:

from construct import *
from vms.rms.IndexedFile import IndexedFile

class MsgRec:
def __init__(self, id, msg):
self.id = id
self.msg = msg

class MsgFil(IndexedFile):
def __init__(self, fnm):
IndexedFile.__init__(self, fnm, Struct('msgrec', SNInt32('id'),
String('msg', 100, padchar=' ')))
def primary_keynum(self):
return 0

db = MsgFil('msgs.isq')
db.put(MsgRec(5, 'Hi from Python!'))
for msg in db:
print(msg.msg)

Arne

[Arne Vajhøj]

On 8/28/2022 4:52 PM, Jan-Erik Söderholm wrote:
> Den 2022-08-28 kl. 20:35, skrev Arne Vajhøj:
>> On 8/28/2022 5:07 AM, Jan-Erik Söderholm wrote:
>>> Den 2022-08-28 kl. 03:13, skrev Arne Vajhøj:
>>>> JFP VMS Python 2.7.18 on VMS Alpha.
>>>>
>>>> Has anyone gotten vms.rms.IndexedFile working?
>>>>
>>>> I am stuck at:
>>>>    NotImplementedError: IndexedFile.primary_keynum
>>>>
>>>> And a quick look at the IndexedFile __init__ and
>>>> primary_keynum make it look difficult to avoid.
>>>

>>> I have used vms.rms.IndexedFile for many years in routines
>>> that reads RMS indexed files and loads Rdb tables.
>>>
>>> But primarily for sequential access, not using key lookup.

It is called in the constructor so it is also called
for sequential access.

>>> I can see that I have this lines, not sure what they does,
>>> probably just added from some RMS example from the
>>> (now closed) JPF PythonVMS web page.
>>>
>>>      def primary_keynum(self):
>>>          return 0
>>>
>>> The rest of the code never calls primary_keynum() and I have
>>> no idea what happens if that is removed…

The exception in the subject line.

>> So you create a class that inherits from IndexedFile and overrides
>> primary_keynum?
>
> If you say so, I guess I do... :-)

I understand *what* the method is doing now.

Demo:

$ typ cre.pas
program cre(input, output);

type
data = record
f1 : [key(0)] integer;
f2 : [key(1)] integer;
end;
database = file of data;

var
db : database;
d : data;

begin
open(db, 'demo.isq', new, organization := indexed, access_method :=
keyed);
d.f1 := 1;
d.f2 := 9;
db^ := d;
put(db);
d.f1 := 2;
d.f2 := 8;
db^ := d;
put(db);
d.f1 := 3;
d.f2 := 7;
db^ := d;
put(db);
end.
$ pas cre
$ lin cre
$ run cre
$ typ lst0.py

from construct import *
from vms.rms.IndexedFile import IndexedFile

class DataFil(IndexedFile):
def __init__(self, fnm):
IndexedFile.__init__(self, fnm, Struct('data', SNInt32('f1'),
SNInt32('f2')))
def primary_keynum(self):
return 0 # <---- note this

db = DataFil('demo.isq')
for d in db:
print('%d %d' % (d.f1, d.f2))
$ python lst0.py
1 9
2 8
3 7
$ typ lst1.py

from construct import *
from vms.rms.IndexedFile import IndexedFile

class DataFil(IndexedFile):
def __init__(self, fnm):
IndexedFile.__init__(self, fnm, Struct('data', SNInt32('f1'),
SNInt32('f2')))
def primary_keynum(self):
return 1 # <---- note this

db = DataFil('demo.isq')
for d in db:
print('%d %d' % (d.f1, d.f2))
$ python lst1.py
3 7
2 8
1 9

Sorry for using Pascal and not Cobol to create
the file, but I am much faster writing Pascal than
Cobol. The Cobol version would use:

... RECORD KEY IS F1 ALTERNATE RECORD KEY IS F2.

I don't understand *why* it works that way. I would have
had it as a ctor argument with default value 0 instead
of having to override a method.

But then I am not a Python person and it would
require 25 volumes of thick books to describe all
that I don't understand.

:-)

Arne

[Arne Vajhøj]

On 8/28/2022 4:53 PM, Jan-Erik Söderholm wrote:
> Den 2022-08-28 kl. 22:52, skrev Jan-Erik Söderholm:
>> Den 2022-08-28 kl. 20:35, skrev Arne Vajhøj:
>>> I will have to try that.
>>

>> I can mail you my complete script if you like.
>> Reads an RMS file and writes to a Rdb table.
>
> He he, just saw your email. I'll reply to that...

I got it working with help from your example.

Thanks.

Arne

[Arne Vajhøj]

And I got into more problems that I had to solve. Specifically
about VAX floating point and Cobol packed decimals in the
records.

Here is my solution in case anyone need to do something
similar.

Record structure is defined in Cobol as:

ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT OPTIONAL D-FILE ASSIGN TO "TESTDB.ISQ" ORGANIZATION IS
INDEXED ACCESS MODE IS DYNAMIC RECORD KEY IS D-ID.

DATA DIVISION.
FILE SECTION.
FD D-FILE.
01 D-RECORD.
03 D-ID PIC X(8).
03 D-IV PIC 9(8) COMP.
03 D-ZV PIC 9(5)V9(2) PACKED-DECIMAL.
03 D-XV COMP-2.
03 D-SLEN PIC 9(4) COMP.
03 D-SV PIC X(50).

Compiled with /FLOAT=G_FLOAT.

To get that working with Python:

from construct import *
from vms.rms.IndexedFile import IndexedFile

from decimal import *

from extras import *

class Data:
def __init__(self, id, iv, zv, xv, sv):
self.id = id
self.iv = iv
self.zv = zv
self.xv = xv
self.slen = len(sv.rstrip())
self.sv = sv.ljust(50)

class DataFil(IndexedFile):
def __init__(self, fnm):
IndexedFile.__init__(self, fnm, Struct('data', String('id', 8),
SNInt32('iv'), PackedDecimal('zv', 5, 2), VAXGFloat('xv'),
UNInt16('slen'), String('sv', 50)))
def primary_keynum(self):
return 0
def pack_key(self, keynum, keyval):
return keyval
def keyval(self, rec, keynum):
return rec.id

Where extras.py contain some extra stuff that I needed to add
(building on the JFP stuff that VMS Python comes with). Copy
attached below.

Arne

====

from ctypes import CDLL, c_double, c_char_p, cast, byref
from construct import Adapter, Bytes
from vms.rms import BCD2Tuple, Tuple2BCD
from decimal import Decimal

CVT_K_VAX_F = 0
CVT_K_VAX_D = 1
CVT_K_VAX_G = 2
CVT_K_VAX_H = 3
CVT_K_IEEE_S = 4
CVT_K_IEEE_T = 5
CVT_K_IBM_LONG = 6
CVT_K_IBM_SHORT = 7
CVT_K_CRAY = 8

librtl = CDLL('LIBRTL.EXE')
lib_convert_float = getattr(librtl, 'CVT$CONVERT_FLOAT')

def float_to_python(val, typ):
res = c_double()
lib_convert_float(cast(val, c_char_p), typ, byref(res),
CVT_K_IEEE_T, 0)
return res.value

def float_from_python(val, typ):
res = '12345678'
lib_convert_float(byref(c_double(val)), CVT_K_IEEE_T, cast(res,
c_char_p), typ, 0)
return res

class VAXFFloatAdapter(Adapter):
def _decode(self, obj, context):
return float_to_python(obj, CVT_K_VAX_F)
def _encode(self, obj, context):
return float_from_python(obj, CVT_K_VAX_F)

def VAXFFloat(name):
return VAXFFloatAdapter(Bytes(name, 4))

class VAXDFloatAdapter(Adapter):
def _decode(self, obj, context):
return float_to_python(obj, CVT_K_VAX_D)
def _encode(self, obj, context):
return float_from_python(obj, CVT_K_VAX_D)

def VAXDFloat(name):
return VAXDFloatAdapter(Bytes(name, 8))

class VAXGFloatAdapter(Adapter):
def _decode(self, obj, context):
return float_to_python(obj, CVT_K_VAX_G)
def _encode(self, obj, context):
return float_from_python(obj, CVT_K_VAX_G)

def VAXGFloat(name):
return VAXGFloatAdapter(Bytes(name, 8))

class PackedDecimalAdapter(Adapter):
def __init__(self, byts, befdec, aftdec):
Adapter.__init__(self, byts)
self.befdec = befdec
self.aftdec = aftdec
def _decode(self, obj, context):
return Decimal(BCD2Tuple(obj, self.aftdec))
def _encode(self, obj, context):
return Tuple2BCD(obj.as_tuple(), self.befdec, self.aftdec)

def PackedDecimal(name, befdec, aftdec):
return PackedDecimalAdapter(Bytes(name, (befdec + aftdec + 2) / 2),
befdec, aftdec)