I noticed this in a program I was working on....
READVU BILL.AMT FROM ADCREDINFO,KEY,39 ELSE BILL.AMT = 0
BILL.AMT<1,1> += DIFF
WRITEV BILL.AMT ON ADCREDINFO,KEY,39
That got me tut-tutting because the writev implies another read
anyway, so we're reading that record and that group that it is in
TWICE.
So I started thinking about something like...
READU ADREC FROM ADCREDINFO,KEY ELSE ADREC = ""
BILL.AMT = ADREC<39,1>
BILL.AMT += DIFF
ADREC<39,1> = BILL.AMT
WRITE ADREC ON ADCREDINFO,KEY
But then I thought about it, and wondered if that would make such a
difference anyway.
Surely when the WRITE is executed we again read the group into memory,
update the group and write it away again.
Long ago I was taught that this kind of READV.... WRITEV.... is
wasteful in terms of disc I/O. But now I'm not so sure.
VLIST doesn't shed much light.
The READVU... WRITEVU generates
0B1CE : 274 reapvu FILEVAR [19] KEY 39 => BILL.AMT
0B1D8 : 0C8 jumpt 0B1E6:
0B1E0 : 0F8 move 0 => BILL.AMT
0B1E6 : 060 dyn_extract BILL.AMT 1 1 0 => $R607
0B1F2 : 004 add $R607 DIFF => $R608
0B1FA : 064 dyn_replace BILL.AMT 1 1 0 $R608 => BILL.AMT
0B208 : 212 writev BILL.AMT FILEVAR [19] KEY 39
The READU.... WRITEU generates
0B212 : 270 reapu FILEVAR [19] KEY => ADREC
0B21A : 0C8 jumpt 0B226:
0B220 : 0F8 move "" => ADREC
0B226 : 060 dyn_extract ADREC 39 1 0 => BILL.AMT
0B232 : 004 add BILL.AMT DIFF => BILL.AMT
0B23A : 064 dyn_replace ADREC 39 1 0 BILL.AMT => ADREC
0B248 : 208 write ADREC FILEVAR [19] KEY
So again it comes down to the amount of I/O that a writev and a write
require.
There is another consideration here. We have thousands of programs,
not all very well written. If we want to know which programs update a
particular attribute, we use a finder paragraph to search for the item
name and attribute number. Ex: "FIND BP CUSTOMER.ITEM(39)
CUSTOMER.ITEM<39> CUSTOMER.ITEM<39,". Of course we then have to look
for CUSTOMER.REC, CUST.REC, CUSTREC, CUST, CUS, ad nauseum. My
favorite (not) is just REC. Yep, we have that, too. Over time we've
identified most all the different ways our major files are referenced.
The real bear is WRITEV. Since the file handle and ID could be called
anything, it's very difficult to locate updates programmatically.
Whether READV and WRITEV are more efficient by microseconds or are
less, for us they are the pits.
I'd welcome any solutions that don't involve poultry sacrifice or
dancing nude in the moonlight (you REALLY don't want to see that).
Regards,
Charlie Noah
Inland Truck Parts
As far as I am aware (and I am open to argument), none of the
multivalue platforms is clever enough to spot that something like
READV F2 FROM FVAR, ID, 2 ELSE ...
READV F7 FROM FVAR, ID, 7 ELSE ...
could actually do just one read and extract the two fields. It does,
of course, potentially gain from record/group level caching where this
is available.
As a general rule, if you want to extract two or more fields, do the
READ and extract the fields yourself.
The one place where READV is really valuable is with field 0 to
determine whether a record exists. Although this still has to go to
the file to look for the record, it doesn't actually read it into
memory. This becomes really important with huge records, especially
those sometimes found in directory type files that might be many
megabytes long.
Understanding WRITEV is even more important. A WRITEV is effectively a
READ followed by a field replacement and a WRITE. This is fine for a
single use but multiple WRITEVs become very inefficient.
Martin Phillips, Ladybridge Systems
READV X1 FROM F.CUST,CUST.ID,21 ELSE X1 = 0
READV X2 FROM F.CUST,CUST.ID,22 ELSE X2 = 0
READV X3 FROM F.CUST,CUST.ID,23 ELSE X3 = 0
READV X4 FROM F.CUST,CUST.ID,24 ELSE X4 = 0
*
...some calculations
*
WRITEV X1 ON F.CUST,CUST.ID,21
WRITEV X2 ON F.CUST,CUST.ID,22
WRITEV X3 ON F.CUST,CUST.ID,23
WRITEV X4 ON F.CUST,CUST.ID,24
At some point someone says "well, the last guy used a WRITEV so it
must be okay" and then you end up with stuff like the above crap that
performs badly and doesn't read so hot either.
> There is another consideration here. We have thousands of programs,
> not all very well written. If we want to know which programs update a
> particular attribute, we use a finder paragraph to search for the item
> name and attribute number. Ex: "FIND BP CUSTOMER.ITEM(39)
> CUSTOMER.ITEM<39> CUSTOMER.ITEM<39,". Of course we then have to look
> for CUSTOMER.REC, CUST.REC, CUSTREC, CUST, CUS, ad nauseum. My
> favorite (not) is just REC. Yep, we have that, too.
You'd love this program then. Somewhere up near the top of this
listing it has something like
EQU ADCREDINFO TO FILEVAR(18).
"Martin Phillips" <MartinP...@ladybridge.com> wrote in message
news:e11457ce-2229-489f...@g29g2000yqe.googlegroups.com...
> Actual database access is at the record level. Even though it happens
> deep inside the database engine, READV is effectively equivalent to a
> READ followed by a field extraction.
>
> As far as I am aware (and I am open to argument), none of the
> multivalue platforms is clever enough to spot that something like
> READV F2 FROM FVAR, ID, 2 ELSE ...
> READV F7 FROM FVAR, ID, 7 ELSE ...
> could actually do just one read and extract the two fields. It does,
> of course, potentially gain from record/group level caching where this
> is available.
>
> As a general rule, if you want to extract two or more fields, do the
> READ and extract the fields yourself.
>
> The one place where READV is really valuable is with field 0 to
> determine whether a record exists. Although this still has to go to
> the file to look for the record, it doesn't actually read it into
> memory. This becomes really important with huge records, especially
> those sometimes found in directory type files that might be many
> megabytes long.
<snip>
Do all platforms support READV of attr 0? I'm thinking D3 doesn't, or at
least didn't.
And as far as limiting memory usage by reading attr 0, I believe the entire
record is always read into memory, and just the requested attribute is
copied into process workspace. Maybe it's just a slight improvement after
all.
Ed
> And as far as limiting memory usage by reading attr 0, I believe the entire
> record is always read into memory, and just the requested attribute is
> copied into process workspace.
There is no attribute zero. At least, if I understand correctly, not
on UniVerse. The key is stored separately from the record. When you do
a READ UV finds the key and then finds the record AFTER the key, so
READV with a zero might be the quickest way to test for the presence
of a record. I've heard that it is, but I've never tested it.
Two things:
1) The above EQUATE to a dimensioned array is very efficient for
handling a large number of files in bulk compared to dealing with each
one individually by name. It's usually used in code generators. Same
goes for referencing attributes:
EQU AR.CURRENT.BALANCE TO 43
EQU CUST.LAST.PAYMENT TO 44
EQU CUST.FILE to FILEVAR(18)
FOR FNUM = 1 TO 99
OPEN FNAMES<FNUM> TO FILEVAR(FNUM) ELSE STOP
NEXT FNUM ; * Note 3 lines, not 99
INVOICE<AR.CURRENT.BALANCE> = BAL
WRITEV BAL ON CUST.FILE,CUST.ID,CUST.LAST.PAYMENT
I'm not saying it's pretty. I'm saying it's a valid technique for
specific kinds of work.
2) To find file usage, check mvScan for U2 at brianleach.co.uk.
T
D3 does not support READV/0.
I hope I have this right - I'll defer to someone with a clue...
We need to separate the concept of memory and workspace here.
If you have large items then READV/WRITEV prevent reading the entire
buffer into BASIC workspace. When this happens with a whole item,
BASIC is going to suck in as many frames as it needs to hold the
variable. If you're working with FlashBASIC then this is all done in
memory. But at the DBMS level, all of the data needs to be pulled
from disk into memory anyway (not BASIC workspace) so that it can be
scanned for attribute marks. So while you save something in post-read
frame manipulation, you gain nothing in disk IO access time.
With WRITEV, a smarter MV system will not read the item into workspace
before the update, it will count the number of bytes in the current
attribute on disk and replace the attribute in-line if it matches the
number of new bytes. If the byte count is different then the DBMS
needs to shift frames up or down, pretty much negating the imagined
economy of a WRITEV. Every platform does this differently.
HTH
T
"Tony Gravagno" <address.i...@removethis.com.invalid> wrote in
message news:qbb1m5p0sf2ab631f...@4ax.com...
Bob, the concept of ReadV/0 doesn't mean you're trying to get
information about the item ID. It means you're testing to see if the
item exists. See:
http://www.pickwiki.com/cgi-bin/wiki.pl?UniVerse_Tips_And_Tricks
It's a convention, a trick that people have coded into, but (not that
I've tried it anytime recently) I don't believe D3 supports this
trick. I know this particular trick has caused some sites difficulty
in migration - as use of many non-standard tricks will...
HTH
T
note: I havent tested other versions of MV yet.
Rich
I'm on uniData.
Since translate through attribute 0 returns the
entire record, I suspected that READV would, too.
Here's my test program, GBP TEST:
OPEN 'GBP' TO FV ELSE STOP 201,'GBP'
READV GOO FROM FV,'TEST',0 ELSE GOO = 'NO CAN READV'
CRT CHANGE(GOO,@AM,CHAR(13):CHAR(10))
Anybody wanna guess what the output of this
program might be?
spoiler... answer below...
It produces "1" (without the quotes.)
Maybe it's returning a Boolean?!?
--
frosty
> Here's my test program, GBP TEST:
>
> OPEN 'GBP' TO FV ELSE STOP 201,'GBP'
> READV GOO FROM FV,'TEST',0 ELSE GOO = 'NO CAN READV'
> CRT CHANGE(GOO,@AM,CHAR(13):CHAR(10))
>
> Anybody wanna guess what the output of this
> program might be?
>
> spoiler... answer below...
>
> It produces "1" (without the quotes.)
>
> Maybe it's returning a Boolean?!?
Maybe it's returning the status bit (success/no success) from the
CHANGE statement.
You will then have MEANINGFUL data for YOUR environment .... all for
little more than it took to write your email, and much less time than
it will take me to read the thread :-)
"Ross Ferris" <ro...@stamina.com.au> wrote in message
news:c550ebbd-1949-4782...@g8g2000pri.googlegroups.com...
It depends on what system you are using. If READV doesn't recognise
field zero as being something special, it is likely to return the
entire record (as in Revelation).
UniVerse takes the THEN clause and returns the record id if the record
exists, and takes the ELSE clause and returns a null string if it
doesn't exist.
Unidata appears to return 1 or 0 as the "record" read though their
documentation isn't too clear on what is supposed to happen.
QM behaves like UniVerse though I notice that our documentation
doesn't quite match with this. It will be fixed.
Ultimately, READV fields 0 is implementation specific but, in all
systems that support it, it allows a program to determine whether a
record exists. I cannot say what happens in other systems but QM
certainly does not read the record into memory for this operation,
providing significant performance advantages over a READ when the
record is huge. A quick experiment with an 80Mb record in UniVerse
suggests it does the same.
Martin Phillips, Ladybridge Systems
Nothing worse than modifying an old system and having to change the
DIM statements all over the place.
You can still do equates if you wish.
> It is ridiculous to ever use writev or readv as it leads to chaos.
I agree 100%. Especially since buggy programs can WRITEV a single
attribute to a record that didn't previously exist, leading to
"mystery" data. Plus, these buggy programs will usually WRITEV using
a record ID of 0, an empty string, or some ridiculous key that
includes system delimiters, making it next to impossible to clean up.
Chaos!
> The simple rule to follow is always dimension all items as an array and
> always equate every element of the array to a standard variable then always
> refer to all variables by name.
> Simple bug free and obvious.
Not bad, but this approach causes trouble if you ever need to handle
*two* records from the same file at once. You handle one with the
MATREAD, but what about the other one? The EQUs don't help now, and
you fall back to integer field references. I prefer to let the EQUs
just be numeric, and then always use dynamic arrays:
* in an INCLUDEd file definition item:
EQU TRNS.SUBGROUP.ID TO 10
* In code using the file
THIS.SUBGROUP = TRNS.REC< TRNS.SUBGROUP.ID >
While theoretically slower, in my work I've never seen it make a
measurable difference.
The additional benefit of this approach is that, per the original
poster's issue, you could scan the code for TRNS.SUBGROUP.ID to find
all programs that touch that field. Again, theoretical, because very
few applications that have kept a consistent coding style throughout
their maintenance history.
> I prefer to let the EQUs
> just be numeric, and then always use dynamic arrays:
>
> * in an INCLUDEd file definition item:
> EQU TRNS.SUBGROUP.ID TO 10
>
> * In code using the file
> THIS.SUBGROUP = TRNS.REC< TRNS.SUBGROUP.ID >
>
> While theoretically slower, in my work I've never seen it make a
> measurable difference.
Also, this method can be used for both dimensioned and dynamic arrays.
Another benefit is that, in D3 at least, if you spell the equated
variable incorrectly, it will raise a runtime error as an unassigned
variable.
However, if your variable is EQUated to a dimensioned array variable,
and you spell it incorrectly, it simply becomes a variable
assignment. No error is raised and your dimensioned array ends up
missing data.
--
Kevin Powick
Aside to Tony - The OP to which I am responding appears to be attributed to
you but I suspect that is false. The wordy but informative and accurate
original, original is surely yours but the succinct question about
understanding it is surely not yours. And perhaps should be signed with a
smiley :)
BobJ
"Tony Gravagno" <address.i...@removethis.com.invalid> wrote in
message news:dh47m5lcnqhvlpuc6...@4ax.com...
Agreed.
> The simple rule to follow is always dimension all items as an array and
> always equate every element of the array to a standard variable then always
> refer to all variables by name.
With all due respect, the assumption that dimensioned arrays are more
efficient than dynamic arrays is unfortunately not a universal truth.
Except for this statement:
>The issue of two reads at the same time is of course what
>object oriented programming solves simply.
I'm picking a nit and it's OT from READV/WRITEV. Never mind.
T
"RJ" wrote:
>Aside to Tony - The OP to which I am responding appears to be attributed to
>you but I suspect that is false. The wordy but informative and accurate
>original, original is surely yours but the succinct question about
>understanding it is surely not yours. And perhaps should be signed with a
>smiley :)
>BobJ
>
>"Tony Gravagno" wrote
Following on the original writev topicI actually disagree with using offsets
into dynamc arrays of titems for many reasons not least of which is that
people always remember the wrong one and wreck the file.
Of course I forgot you hate generated code because you went and played with
the output instead of using the proper tool and as a result lost all of your
work. Check your posts from a year or two back.
Peter McMurray
"Tony Gravagno" <address.i...@removethis.com.invalid> wrote in
message news:dh47m5lcnqhvlpuc6...@4ax.com...
On CUBS systems there is (used to be?) a memo file of textual notes;
the last note is appended to the end of the memo record as the last
attribute, and the number of that last attribute is stored in
attribute 1 of the same memo record. CUBS used to use a trick to speed
up (circa R83) adding note lines to this file by using a READV +
WRITEV + WRITEV which was faster than a standard READ + WRITE,
particularly as the number of memo lines increases.
Here are two test programs to demonstrate:
:ct bp memo.test (double writev)
memo.test
001 open "sb.temp" to sb.temp else stop 201,"sb.temp"
002
003 id = "12345"
004 rec = 1
005 t = time()
006 d = date()
007 tot = 10000
008 a0 = @(0)
009
010 write rec on sb.temp,id
011
012 for n = 1 to tot
013 read rec from sb.temp,id else rec = 1
014 pos = rec<1> +1
015 rec<1> = pos
016 rec<pos> = "the quick brown fox jumped over the lazy dog"
017 write rec on sb.temp,id
018 if mod(pos,100) eq 0 then
019 call counter.sub(n,t,d,tot,txt)
020 print a0:n:txt:
021 end
022 next n
023
024 call final.count(n,t,d)
025
:ct bp memo.test2 (single write)
memo.test2
001 open "sb.temp" to sb.temp else stop 201,"sb.temp"
002
003 id = "12345"
004 rec = 1
005 t = time()
006 d = date()
007 tot = 10000
008 a0 = @(0)
009
010 write rec on sb.temp,id
011
012 for n = 1 to tot
013 readv pos from sb.temp,id,1 else pos = 1
014 pos +=1
015 writev pos on sb.temp,id,1
016 writev "the quick brown fox jumped over the lazy dog" on
sb.temp,id,pos
017 if mod(pos,100) eq 0 then
018 call counter.sub(n,t,d,tot,txt)
019 print a0:n:txt:
020 end
021 next n
022
023 call final.count(n,t,d)
024
And the results....
D3 7.5.1 Linux (with MFT drives!)
Regular compile (non-flash)
:memo.test
10001 records processed in 00:31 = 323 per second.
:memo.test2
10001 records processed in 00:24 = 417 per second.
:memo.test
10001 records processed in 00:30 = 333 per second.
:memo.test2
10001 records processed in 00:22 = 455 per second.
You can see why CUBS did it that way.
That was then, this is now; Flash-compiling negates the writev
advantage, I guess because of Flash doesn't change the I/O speed but
it does radically improve the dynamic array (record) handling. Still,
the fact that you can do two WRITEVs in the same amount of time as a
single WRITE implies that WRITEV is quite a bit faster.
Flash-compiled:
:memo.test
10001 records processed in 00:23 = 435 per second.
:memo.test2
10001 records processed in 00:22 = 455 per second.
:memo.test
10001 records processed in 00:23 = 435 per second.
:memo.test2
10001 records processed in 00:23 = 435 per second.
Here are the results from a UV/Linux system (I shortened the loop to
3000 on the UV machine to keep the times similar).
RELLEVEL
X
10.2.7
PICK
PICK.FORMAT
10.2.7
>MEMO.TEST
3001 records processed in 00:16 = 179 per second.
>MEMO.TEST2
3001 records processed in 00:35 = 85.0 per second.
>MEMO.TEST
3001 records processed in 00:16 = 179 per second.
>MEMO.TEST2
3001 records processed in 00:35 = 85.7 per second.
On this UV system, it looks like two WRITEVs are about twice as slow
as a single WRITE, so there does not seem to be any advantage at all
to using WRITEV here.
/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006
> Hi Tony
> Perhaps you should try reading the post to which I was responding which is
> part of my reponse...
Don't worry Tony. Even after this "clarification", I still don't
follow it either.
--
Kevin Powick
Phew, thought it was just me. (Note, I tried, I really tried.)
Perhaps he over-edits his notes and deletes significant words or
sentences.
Perhaps he doesn't read his notes at all before posting to see if they
make any sense - or if they contain factual verifiable details - or if
they contain statements that are easily refuted with a simple Google
search ?
Or perhaps he's prone to visiting this forum after an aperatif and/or
a digestif or two. Personally I wish people wouldn't engage in public
discourse simultaneously with mind altering substances.
Seriously - the first post didn't make sense and then in this last
one, what prompted the outburst about code generation when no one was
talking about that? For the record, BTW, I'm quite fond of code
generators, have blogged about the topic a few times, and spend a good
amount of my daily time writing and using them. I dislike bad code
generators. I don't categorically "hate" code generators - or
anything else.
T
> However the major benefit in even that singular scenario is the use of
> standard variable names.
Customer(name)
Customer<name>
or perhaps even
readv customer_name from file.customer,customer_code,name else ...
Though for the record, I don't recall having ever used a readv/writev
in "real" code (or even in play things come to think of it)
"always equate every element of the array to a standard variable then
always refer to all variables by name"
which means no "shortcuts" like:
*
* Clear address info
*
FOR NA = 1 TO 12
RECORD(NA) = ''
NEXT NA
Also make sure everything is re-compiled when a change is made. And of
course, the data itself will also have to be updated if the schema is
changed.
hth
Colin Alfke
Calgary, Canada
On Jan 29, 1:36 am, "Peter McMurray"