How to read deleteed records from a file?

[Rolf Mittag]

There's (AFAIK) definitely _no_ way to get deleted
records from database management. Makes sense
doesn't it?

So take what was already posted :
MI routine

or wrapped with "wrkDbF".

There's still 1 alternative left :

save the file with the deleted records
now extract the deleted records from the save file
(record length is 512 AFAIK)

HTH Rolf

---------------------------
Dipl.Inf.(FH) Rolf P Mittag
Leipziger Str. 50
D-69214 Eppelheim
Fon: +49 (6221) 76 78 60
Fax: +49 (6221) 76 80 26
eMl: r...@r-m-e-d-v.de

[elb...@cmfurniture.com]

I think there is a way you can chain to a RRN even if it is deleted.

I don't know how.

But the record does remain in the file until a RGZPFM is performed.

I read a warning I think in an IBM V2R1 manual years ago that if you
created a physical file and specified to reuse deleted records. The
problem existed that if you chained to the RRN of a deleted record it
may cause the record to be re-activated.

So they are there and they can be access and re-activated somehow?????

On Fri, 16 Mar 2001 12:26:12 +0100, "Thomas Raddatz"
<thomas....@gfd.de> wrote:

>Unfortunately it is not possible to READ a deleted record by RRN. But it is
>possible to write a new record on a deleted RRN by specifying the RRN
>keyword at the F-spec.
>
>Thomas Raddatz.
>
>
>Ugo Gagliardelli <non.mi.rompe...@con.delle.cagate.org> schrieb in
>im Newsbeitrag: 3AB12EB5...@con.delle.cagate.org...
>>
>>
>> Thomas Raddatz wrote:
>> >
>> > Hi,
>> >
>> > I know that it is somehow possible but I do not know how. So does
>someone
>> > know how to read deleted records from a file?
>> By rrn?
>> --
>> Dr. Ugo Gagliardelli, Modena, Italy
>> Spaccamaroni andate a cagare/Spammers not welcome
>
>

[Jim Langston]

Take a look at DMP (dump object).

Regards,

Jim Langston

[Rolf Mittag]

Did I say database management? Sure I did.
Is dumpobj database management? Sure it isn't.

--

[Thomas Raddatz]

Hi,

I know that it is somehow possible but I do not know how. So does someone
know how to read deleted records from a file?

Thomas Raddatz.

[Eric Chabot]

The simplest way to to install WRKDBF, a freeware available at
www.wrkdbf.com

[Ugo Gagliardelli]

Thomas Raddatz wrote:
>
> Hi,
>
> I know that it is somehow possible but I do not know how. So does someone
> know how to read deleted records from a file?

[Douglas Handy]

Thomas,

>I know that it is somehow possible but I do not know how. So does someone
>know how to read deleted records from a file?

By fancy footwork using MI. Dave McKenzie wrote a nifty freeware
utility to do just that -- UNDEL. It is available in both CISC and
RISC formats in the shareware section at News/400 (
www.as400network.com ). He released the MI source code too.

The same utility is used under the covers to enable un-delete of
records by another freeware utility, WRKDBF. This utility can be
downloaded free from www.wrkdbf.com in either RISC or CISC format.

Doug

[Thomas Raddatz]

In the meantime I found the following information in the "Data management"
guide and I wonder what it is good for if it is not possible to read deleted
records:

¦ ¦ ¦ Bit 4: Deleted record indicator.
¦
¦ ¦ ¦
¦
¦ ¦ ¦ 0 Current file position is at an active
record. ¦
¦ ¦ ¦ 1 Current file position is at a deleted
record. ¦
¦ ¦ ¦
¦

The information is part of the "I/O Feedback Area for Database Files".

Any comments?

Thomas Raddatz.

Thomas Raddatz <thomas....@gfd.de> schrieb in im Newsbeitrag:
3ab0...@news.piro.net...

[Thomas Raddatz]

Unfortunately it is not possible to READ a deleted record by RRN. But it is
possible to write a new record on a deleted RRN by specifying the RRN
keyword at the F-spec.

Thomas Raddatz.

Ugo Gagliardelli <non.mi.rompe...@con.delle.cagate.org> schrieb in
im Newsbeitrag: 3AB12EB5...@con.delle.cagate.org...
>
>

[z01z]

there is the command in DBU too, if you have it, you can try

[Thomas Raddatz]

Thanks to all who participated at that thread. I think that I will try to
mail to Dave McKenzie. I have his great tool RTVCMDSRC and there is an old
e-mail address. Hopefully it is still valid.

Thomas.

Thomas Raddatz <thomas....@gfd.de> schrieb in im Newsbeitrag:
3ab0...@news.piro.net...

[Dave McKenzie]

Hi Thomas,

Thanks for the kind words :-) It's dav...@galois.com

--Dave

[Dave McKenzie]

AFAIK, there is no way to read deleted records using normal data
management, such as in an HLL program (CL, RPG, COBOL, etc.). I know
of two other ways:

1. The method used in the UNDEL utility on
www.as400network.com/sharewarefiles
This involves a system-state MI pgm which has an altered machine
instruction that allows it to read the internal structures of a file
by virtual addresses (source is on the website, for the pathologically
curious). However, the system-state pgm can be a problem because some
managers may not want to allow system-state pgms on their machine.
Even if they do, at security levels 40 and above it may require
changing system values to allow the system-state program to be
restored.

2. Saving the file to a save file, copying the save file to a physical
file (record length 528) and finding the deleted records in the
physical file. This is how the tool in QUSRTOOL works. This
overcomes the problem with system-state pgms, but is of course slower.
If the file is huge, not only will saving it take time, but there may
not even be enough free space on the system to store it! However, to
get around that, you can use CPYF to copy only the deleted record to
another file, then save that file. For example, if the deleted record
is RRN 10:

cpyf mylib/myfile qtemp/filex crtfile(*yes) fromrcd(10) nbrrcds(1)
compress(*no)

Then save filex to a save file and copy the save file to a physical
with record length 528. You can use FTP to do that. If there is only
one record in the saved file, start in record 41 of the 528-byte file
to find it. Each record in a physical file is preceded by a status
byte (in which the leftmost bit says it's an allocated record, and the
second bit is 1 if it's deleted). In record 41 there will be two
records starting at offset x20 (position 33): a record having all
fields with default values (blanks and zeros), and the deleted record.
So to figure the position of the deleted record from the start of
record 41, add 33 + 1 + <record-length of the original file> + 1. The
status byte for the default record will be x80 and xC0 for the deleted
record.

(Finding the deleted record will be a little more complicated if the
file has date, time, timestamp, ALWNULL or variable-length fields or
REUSEDLT.)

Of course, a third method is to use SST to find the deleted record in
the original file, but that's messy and this post is already too long.

--Dave

[Thomas Raddatz]

Thanks a lot for your detailed description. However it seems that both ways are
to complicated for what I need it for.

Here is some background information for you.

In our shop we use Softlanding´s PDQ for copying data between files since we
have some huge files with millions of records. PDQ uses service programs for
converting data in case that the file was changed. In order to test the service
programs prior to the copy process done by PDQ we need a program that reads the
records from the source file, pass the buffer to the service program and write
the buffer to the target file. I already figured out how to connect to a service
program at runtime (QleGetExp) and it works fine. The problem is that PDQ may
pass a parameter COMPRESS(*NO) to the service program depending on the setup. In
that case my test program should also be able to process deleted records. Since
I believe you that it is not possible to read deleted records by a HLL program I
wonder how they do it. Regarding to the security level 40 problems I do not want
to assume that they use a MI routine for that. Also a SavF would be a bad choice
due to the amount of records.

By the way, do you know how to access the null map buffer of a record? I only
found information about the null map of the key fields in the file information
data structure.

Thomas Raddatz.

Dave McKenzie schrieb:

--
*=====================================================*

e-mail Adresse:
Thomas....@tools400.de

Homepage:
http://www.tools400.de

*=====================================================*

[Dave McKenzie]

Oh, I think I see. To paraphrase Bill, "It depends on what the
definition of 'read' is."

The methods I described can be used to retrieve the contents of
deleted records, but it sounds like PDQ only detects the *presence* of
deleted records, not their contents, and puts deleted records in the
same position in the target file, but not necessarily containing the
same data as the original records. After all, who cares what data
they contain, because your applications won't be able to "read" the
data from them anyway.

If you are reading the records yourself in RPG using READ in arrival
sequence, you can look at the RRN field in the file information data
structure. If it jumps by more than one from one record to the next,
you can assume that the missing records are deleted. If you use CHAIN
by RRN, there's a "deleted record indicator" bit in the file
information data structure (position 386) that tells you if the record
is deleted when you get a "not found" indication. If you write
records in RPG, one way to write a deleted record would be to write an
active record (who cares with what data), then delete it.

If you need more information about what PDQ does, maybe Mark Phippard
can tell you; I think he works for Softlanding.

BTW, it appears that the null map isn't available to RPG. Evidently
you have to use the %NULLIND function on each field to see if it's
null.

--Dave

On Mon, 19 Mar 2001 21:02:24 +0100, Thomas Raddatz
<Thomas....@online-club.de> wrote:

>Thanks a lot for your detailed description. However it seems that both ways are
>to complicated for what I need it for.
>
>Here is some background information for you.
>

>In our shop we use Softlanding愀 PDQ for copying data between files since we

>have some huge files with millions of records. PDQ uses service programs for
>converting data in case that the file was changed. In order to test the service
>programs prior to the copy process done by PDQ we need a program that reads the
>records from the source file, pass the buffer to the service program and write
>the buffer to the target file. I already figured out how to connect to a service
>program at runtime (QleGetExp) and it works fine. The problem is that PDQ may
>pass a parameter COMPRESS(*NO) to the service program depending on the setup. In
>that case my test program should also be able to process deleted records. Since
>I believe you that it is not possible to read deleted records by a HLL program I
>wonder how they do it. Regarding to the security level 40 problems I do not want
>to assume that they use a MI routine for that. Also a SavF would be a bad choice
>due to the amount of records.
>
>By the way, do you know how to access the null map buffer of a record? I only
>found information about the null map of the key fields in the file information
>data structure.
>
>Thomas Raddatz.
>
>Dave McKenzie schrieb:
>>
>> AFAIK, there is no way to read deleted records using normal data
>> management, such as in an HLL program (CL, RPG, COBOL, etc.). I know
>> of two other ways:
>>

>> <snip>

[Thomas Raddatz]

Dave,

It seems to be a good idea to read the file in arrival sequence and watch the
RRN. I will think about it.

Thanks,

Thomas.

Dave McKenzie schrieb:

>
> Oh, I think I see. To paraphrase Bill, "It depends on what the
> definition of 'read' is."
>
> The methods I described can be used to retrieve the contents of
> deleted records, but it sounds like PDQ only detects the *presence* of
> deleted records, not their contents, and puts deleted records in the
> same position in the target file, but not necessarily containing the
> same data as the original records. After all, who cares what data
> they contain, because your applications won't be able to "read" the
> data from them anyway.
>
> If you are reading the records yourself in RPG using READ in arrival
> sequence, you can look at the RRN field in the file information data
> structure. If it jumps by more than one from one record to the next,
> you can assume that the missing records are deleted. If you use CHAIN
> by RRN, there's a "deleted record indicator" bit in the file
> information data structure (position 386) that tells you if the record
> is deleted when you get a "not found" indication. If you write
> records in RPG, one way to write a deleted record would be to write an
> active record (who cares with what data), then delete it.
>
> If you need more information about what PDQ does, maybe Mark Phippard
> can tell you; I think he works for Softlanding.
>
> BTW, it appears that the null map isn't available to RPG. Evidently
> you have to use the %NULLIND function on each field to see if it's
> null.
>
> --Dave

--

[Tom Liotta]

Dave McKenzie wrote:
>
> 2. Saving the file to a save file, copying the save file to a physical
> file (record length 528) and finding the deleted records in the
> physical file. This is how the tool in QUSRTOOL works. This
> overcomes the problem with system-state pgms, but is of course slower.
> If the file is huge, not only will saving it take time, but there may
> not even be enough free space on the system to store it!

I have not tried it, but the recent Save to Application (QaneSava) API
along with the Save to Application Exit Program might get around the
need to have space for a complete save file on the system. V4R3 and
above.

Tom Liotta

--
Tom Liotta
PowerTech Group, Inc.
19426 68th Avenue South
Kent, WA 98032
Phone 253-872-7788
Fax 253-872-7904
http://www.400Security.com

[Dave McKenzie]

Thanks, Tom. I wasn't aware of that API. I tried it and it does
eliminate the need for a huge save file. Curiously, it creates a save
file in QTEMP of the QSYSWRK job that the exit pgm runs in, but when I
tried it with a 200 Meg file, that save file never got bigger than
about 60K. It seems to act like a bucket with a hole in it.

Of course, you still have to spend the time to read thru the save data
blocks sequentially til you get to the deleted record you want. Maybe
on some future release they'll add a STARTAT parm :-)

--Dave

On Wed, 21 Mar 2001 05:23:05 GMT, Tom Liotta <tho...@inorbit.com>
wrote: