PDOX/BDE CORRUPT FILE OTHER THAN HEADER 2,3 times a day...

[Clinton R. Johnson]

I run a multiuser database, and for years, it ran fine. Multiple applications share r/w access to a common set of tables.

Several months ago, the database suddenly started experincing corruptions. No software or hardware installions co-incided with the corruption. All network cards are been replaced, all machines have been reinstalled, and a low level network diagnostic reports NO packet corruptions. No other applications experience corruption.

Now, it experiences DAILY (more than in fact, it corrupts 2, 3 times a day) corruptions.

The errors range from "CORRUPT FILE OTHER THAN HEADER' to unreported corrupt indexs..

Originally, it was simple enough to copy all the data from the old table into a pristine copy of the table (via PACK, or record by record copying), but eventually, records started going missing. Now I have to use TUTILITY based utilities (Borland's and Pdxrbld.exe from RK Solutions) to recover the tables.

Records started going missing AFTER I start faithfully calling dbiSaveChanges.

With the exception of entire records missing, no records are actually damaged.

As well as corrupt file other than header, the index files seem to corrupt. When using secondary indexes with ranges OR SQL statements with use the secondard index, data which is IN the table simple does not appear in the result set - either some or all records are missing.

All machines have BDE 5.11 installed, run norton antivirus, there is a mix of NT4 workstation, W2K professional, and Win98. Files are served from a NT4 server.

BDESupport reports that disabling oplocks can resolve frequent corruption, but include statements from Borland that they have never found that oplock tweaking makes and difference in performance. I am also concerned that turning off a basic network perfomance optimization will result in drastically reduced perfomance.

Has anyone else experienced similar problems and found a solution?

Can anyone comment of the performance loss from turning off oplocks?

Where possible, please CC any response to b...@xepol.com

Thanks.

- CLinton R. Johnson

[Bill Todd (TeamB)]

Make sure that Local Share is on on _every_ machine and that _every_ machine
uses the same path to the network control directory (NetDir in BDE
Administrator) and the same path to the database directory. If that does not
cure the problem try turning opportunistic locking off. I have not seen a
performance loss from thsit but I have not run benchmarks on a heavily
loaded server.

--
Bill

[Lutz Weder]

We got the same problem with Paradox 8 and Paradox file format. Below we have an example to create a corrupt Paradox table.

Under certain conditions Paradox tables can become corrupt during normal operations (without
user or power shutdown, even on a standalone machine - no network!!!). The error occurs often at a large number of records (100,000 and more) and it depends on the structure of the record, index structure, file size, Paradox blocksize and level. Unfortunately knowbody can determine it correct. Browsing in tableview we get the
message "File other than header".

The reason is: the primary index file PX is invalid. We have discussed that problem detailed in the Corel Newsforum Paradox 8 title "Corrupt Paradox Tables", author Lutz Weder, 08-Febr-2000 and in the Newsforum borland.public.bde, author Uwe Thiel and Lutz Weder, 06-April-2000.

If the tables become corrupt no TCursor command works correct (setRange, locate, moveToRecord).

Please contact Bill Todd, a member of TeamB. We discussed the problem and he could reproduce
this bug!

We gave a bug report to borland (#435008) in April 2000 with the examle below but NOTHING is fixed, although they did concern, that the could reproduce this bug.

That a DBMS like the BDE can produce corrupt index files during normal operations without power or user shutdown even on a standalone machine is the worst case for that DBMS and it should be a shipstopper for it!!!

Here again the example to create a corrupt table "art2.db" with an invalid primary index file:
(tested on WIN95/98/NT even with Paradox 9 on Linux with the same result, BDE versions 4, 5 and 5.11 the same)

____Important the BDE settings: level 5, blocksize 4096, BDE version 5.11______

This settings must be __before__ creating our example table!

The table structure is like following:

MANDANT S *
ARTNR A 20 *
LIEFERANT A 4 *
KURZTEXTA A 40

With level 7 and blocksize 32kb this example works, but other tables causes trouble.
We can give you another example to create a corrupt table in level 7 and blocksize 32kb.

In the small Paradox programm below we fill the table with 328,031 records (the number where a customer of us
get problems). Then we open it in tableview with Paradox 8 or in the Database Desktop in Delphi 5.
Browse the table. Push the button of the vertical scrollbar and move it down. In the status bar you
see the message "Scrolling 1 - 1". At about 80% down the message is "Scrolling - ". Release the
button and all shown records will disappear and you got the message "Corrupt file - other than header".

With a hexeditor we found, that the PX file is wrong: there are 2 invalid blocks with empty and
invalid values. In the Paradox 8 newsforum is a detailed article from us (22-Febr-2000) with an
appendix of a document (DOK6engl.doc) where we describe that detailed. There are also screenshots
of the hexeditor, which show the invalid index file. __We can send you this document per email - if you want.__

Also the secondary index files can become invalid. We have also examples!

Here is the Paradox programm to fill the table, we tested it with a similar Delphi 5 programm with the same result!

method pushButton(var eventInfo Event)
var
tc TCursor
i LongInt
endvar

tc.open("c:\art2")
tc.edit()

for i from 10000001 to 10328031 ;; the field artnr should have always 8 digits
message(i)
tc.insertRecord()
tc.mandant = 1
tc.artnr = string(i)
tc.lieferant = "ABCD"
tc.kurztexta = "1234567890123456789012345678901234567890"
endfor
msgInfo("","328031 records added!")
endMethod

Although we discussed that problem already a year ago there is no solution or workaround from Borland or from anyone.

Normally this dramatic bug should forbid the use of the BDE for commercial database applications with large tables.
This bug should produce an outcry from all professional developers which use the BDE with Paradox file format - but we are wondering: nothing happens.

If you need some informations contact me direct: in...@wedersoft.de.

Regards from another "poor pig" that uses Paradox with the BDE.

"Clinton R. Johnson" schrieb:

[David R. Robinson]

>> the database suddenly started experincing corruptions. No software
or hardware installions co-incided

Sorry you are experiencing this. IMHO, the real solution is to get off
of Paradox. Obviously that is not something you can do quickly, but
you'll be better off in the long run if you get off of Paradox. It's
simply too unstable IMHO.

We had a similar situation with a customer. They had over 100 users
using Paradox. Our app supports Paradox (we strongly discourage
anything other than single user use with Paradox), InterBase, Oracle, MS
SQL Server, Sybase, and Informix but since it was free, they used
Paradox. They used it for several months and we had no idea that they
had over 100 users on Paradox. When we found out they were using
Paradox, we begged them to get on a real client/server engine. They
told us that they weren't having any problems and ignored our please.
Until... All of the sudden, once the database got to a certain size,
the DAT file would get truncated to 128 bytes (i.e. 100% of the data was
lost and they had to revert to backup). After this happened about 5
times, they finally believed us and switched to another database engine.

I'd suggest looking at InterBase. You can get it for free if you want
the open-source version. You can use Jason Wharton's IBObjects to
convert your Pdox app to IB fairly quickly. He's done a lot of work to
make converting from Pdox to IB easy. His website is www.ibobjects.com.

--
David R. Robinson
InterBase Installation Info - http://ibinstall.defined.net

[Clinton R. Johnson]

a) You can not run with different netdir's, applications just will not run.
b) all my applications check the localshare true before starting up.

Why Borland even makes this switch available is a question, why the ship the BDE with it set to false, which makes even single station applications fail is an even better question

But anyone that has had a multi user system running even for an hour has figure this much out.

- Clinton R. Johnson

[Clinton R. Johnson]

Thanks. I've been trying to convince my customers to move over for 2 months... it would appear that they have finally decided to move over (thankfully)

At least, they've bought the tools, now I have to convince them to redivert my attention from another project (which CAN wait) to porting to Interbase (which obviously cannot wait)

- Clinton R. Johnson

[Bill Todd (TeamB)]

Remember that Paradox tables date back to 1985 and in those days no one had
ever heard of multi-user access to a PC database. Local area networks were
interesting experimental toys then but few were using them in a production
environment. Suffice it to say that there were good reasons for adding Local
Share at the time it was added (around '87 if my memory is correct) and for
making the default False. It is false today because, like all software
companies, Borland worships the god of backward compatibility for fear that
if they change the default they will get 10 million support calls from angry
users whose apps no longer work the way they expect.<g>

--
Bill

[Clinton R. Johnson]

In article <3b49273f$1_1@dnews>, bill....@dbginc.com says...

Except that turned on, the BDE still works right (perhaps a little slower for those still foolish enough to run novel networks, as if the 4.0 fiasco wasn't lesson enough!), without it, the BDE fails to run right even with multiple application instances on 1 machine.

Backwards compatibility to buggy behaviour is probably not a good thing...

- CLinton R. Johnson

[Joe Griffin]

> without it, the BDE fails to run right even with multiple application instances
on 1 machine.

I disagree!

Back in 1994 at one of the London Paradox Conferences, Kevin Smith (of Softbite)
did a couple of sessions on Paradox locking; concluding with 10 dos and don'ts of
locking. I took notes at the time and wrote them up as my "bible". Since then,
I've had 5 applications running under PxWin 4.5 (no, they never wanted to upgrade,
as none of the apps was "broken") at the London Air Traffic Centre. For all those,
where the data is held on the network server, "Local Share" is OFF.

My notes from the time state:

"Don't use "Local Share = TRUE" unless absolutely necessary.

On peer-to-peer networks, or if using multiple applications simultaneously on one
machine, the ODAPI (or IDAPI) setting of "Local Share" should be set to "TRUE".
This setting ensures that the local hard disk is locked as required. For
file-server networks, this is not necessary; no other user can access your hard
disk, so turn Local Share off, for faster response. (Note that Paradox's ODAPI
file will handle multiple sessions on one machine, so locking is only required if
more than one application is running and they are not all ODAPI tasks.)"

... Joe
Member of the UK Borland User Group

[Antony Hoon]

"Joe Griffin" <JoeGr...@nospam.cix.co.uk> wrote in message
news:VA.0000000...@nospam.cix.co.uk...

> Back in 1994 at one of the London Paradox Conferences, Kevin Smith
(of Softbite)
> did a couple of sessions on Paradox locking; concluding with 10 dos
and don'ts of
> locking. I took notes at the time and wrote them up as my "bible".

I'd be glad to take a peek into your 'bible'. Can you please share
that section?

--
Antony Hoon
Delphi Addicted
(Replace 4x7 with 7777 to reply..)

[Joe Griffin]

In article <3b4a46d9_2@dnews>, Antony Hoon wrote:
> Can you please share that section?

You have mail.