Estimating the potential increase in the area size of a DMSII structure due to a delete

[Luke Numrych]

The "Enterprise Database Server DASDL Programming Reference Manual" alludes that "new areas can also be allocated as the result of delete operations". Unfortunately, no further information is given as to when can that happen and under what conditions, or exactly how are those new areas created (since one hopes it is not on a 1-to-1 basis...).
Can anyone point me to where I could find more information about that?
Is there a way to estimate the potential increase in the area size of a DMSII structure that can be caused by a delete operation based on the number of records to be deleted?

[Luke Numrych]

I may have answered my own question... the data set in question is a STANDARD data set, and I see in the description of the "AREAS" option that "enough available space is allocated to avoid run-time limit errors even if all records in the data set are deleted".
It I understand this correctly, I should not have to worry then?

[Paul Kimpel]

No, you normally don't need to worry about the extra space sometimes
required to manage deleted records. It's only a problem if the data set
is near the maximum number of areas declared/computed for it in DASDL,
or if POPULATIONINCR is in effect for the data set, it is near the
maximum 1000-area limit.

If you look in DASDL manual Appendix C, "Standard Data Set (Fixed
Format)", you can see the mechanism being used. In a Standard Fixed
Format data set, each block of data records occupies a fixed number of
disk sectors and contains a fixed number of records. The first (and
usually largest) part of the file consists of blocks that have (or at
one time had) data records.

Following that, if any records have been deleted and not reused by new
records, is a contiguous set of segments called the DKTABLE. This is a
list of the locations of deleted records within the data blocks.
Following the DKTABLE (which can grow dynamically) there may be
allocated space that has not yet been used. The last segment of
allocated space holds the LASTRECORD structure, which among other things
describes the location and size of the DKTABLE.

If all you do is add records to a data set, there is no DKTABLE, and
LASTRECORD will indicate that. When you delete a record, the
ACCESSROUTINES marks it as deleted by writing NULL values in all of the
record's REQUIRED items. This allows FIND NEXT operations on the data
set to skip over deleted records. It then appends the AA word for that
record (an internal pointer to a record, also described in Appendix C)
to the DKTABLE, creating the DKTABLE immediately after the last in-use
block if necessary. If the DKTABLE has filled up to the end of the
currently-allocated space for a file, it can spill over into a
newly-allocated area.

The DKTABLE is used LIFO. When you add a record to a data set, the
ACCESSROUTINES first checks the DKTABLE. If there are entries, it takes
the last one and reuses the record that entry points to, popping that
entry from the DKTABLE. If the DKTABLE is empty or does not exist (same
thing, really), the ACCESSROUTINES adds a new data block at the end of
the existing data blocks, allocating a new area for the file if necessary.

The times you will see the DKTABLE really grow (and probably cause new
areas to be allocated) is when you delete lots of records. In the
extreme case, if you delete all of the records in the file, you will end
up with the data portion of the file filled with records marked as
deleted, and the DKTABLE having an entry for every record in the data
area of the file.

If you have deleted a large number of records from a data set and don't
plan to add them all back, you might want to consider doing a reorg,
which will consolidate all of the in-use records and eliminate the
DKTABLE. Otherwise, as you add new records they will reuse the deleted
slots and the DKTABLE will shrink until all of the deleted slots have
been reused, after which new data blocks will overwrite the area
formerly occupied by the DKTABLE, and eventually cause new areas to be
added to the file.

Paul

[Luiz Arroyo]

The DK table is a list of deleted records entries that can be reused later. To store this information, space is needed. If you delete a lot of records, extra areas can be allocated to store the DK table entry for the new deleted records. If you are near 1000 areas for a specific dataset with millions of records to delete the program might receive a Limit Error exception on the DELETE statement.