Error DBFCDX/1002 Open error: xxx.dbt (DOS Error 2)

[Claudia Neumann]

Hi all,

on opening xxx.dbf with errors in the database with

dbusearea(.f.,"DBFCDX",cdbf,calias,.t.,.f.)

I get the error

Error DBFCDX/1002 Open error: xxx.dbt (DOS Error 2)

Where does the dbt come from?

Best regards

Claudia

[francesco perillo]

Is there a memo field in the dbf ?

[Claudia Neumann]

Sorry, forgot:

cdbf := "xxx.dbf"

[Claudia Neumann]

Is missing as well.

But missing xxx.fpt should give DBFCDX/1010 error

Best regards

Claudia

[Jonce]

To Check the dbf file if created by "DBFNTX" with memo filed.

You can't force a dbf with dbt file to be opend by "DBFCDX"

You can test to create and open a dbf with memo filed with different driver by too as DBX or DBU.

But you can open it by "DBFNTX" then copy to a new file  by "DBFCDX"

Jonce.

2012/9/15 Claudia Neumann <dr.claudi...@gmx.de>

[Jonce]

sorry 

"by tools like dbx.exe or dbu.exe ...etc."

2012/9/15 Jonce <jonc...@gmail.com>

[Claudia Neumann]

Hi all,

dbu.exe (Clipper dbu.exe and Harbour dbu.exe) gave the same error.

Thank you for the hint to use the "DBFNTX" driver. And the error message was
right: the dbt file was missing.

With an empty dbt file I now succeeded to open the old dbf files with "DBFNTX"
and could convert them to "DBFCDX" files. What a mess with old data =8-/.

Many thanks

Claudia

[Przemyslaw Czerpak]

On Sat, 15 Sep 2012, Claudia Neumann wrote:

Hi Caludia,

> dbu.exe (Clipper dbu.exe and Harbour dbu.exe) gave the same error.
> Thank you for the hint to use the "DBFNTX" driver. And the error message was
> right: the dbt file was missing.
> With an empty dbt file I now succeeded to open the old dbf files with "DBFNTX"
> and could convert them to "DBFCDX" files. What a mess with old data =8-/.

I have to comment it because above is completely false information
which can confuse other users.
All Harbour DBF* RDDs can work with any memo files and automatically
recognize type of memo file (DBT, FPT, SMT). It means that you do not
need any conversions or use DBFNTX to open files with DBT memos.
You can use DBFFPT, DBFCDX, SIXCDX, DBFNSX and they will work without any
problems. Please also not that the error message was correct from the
beginning. DBT file was missing. It was enough to not ignore it to
resolve the problem.

BTW when DBF file is open the Harbour DBF* RDDs automatically recognize
type of memo file reading DBF signature from DBF header.
For newly created files by dbCreate() you can set default memo type using:
rddInfo( RDDI_MEMOTYPE, <nType> [, <cRdd> ] )
where <nType> is type DB_MEMO_DBT, DB_MEMO_FPT or DB_MEMO_SMT, i.e.:
rddInfo( RDDI_MEMOTYPE, DB_MEMO_SMT "DBFCDX" )

best regards,
Przemek Czerpak

[Claudia Neumann]

Hi Przemek,

You are right.
What was confusing me was the error message:

DBFCDX/1002 Open error: xxx.dbt (DOS Error 2)

Shouldn't it be:

DBFNTX/1002 Open error: xxx.dbt (DOS Error 2)
?

Kind regards

Claudia

[Qatan]

Hello Przemek,

>You can use DBFFPT, DBFCDX, SIXCDX, DBFNSX and they will work without any
>problems.

I just would like to understand a bit what is the difference between
DBFNSX and SIXCDX, please. Are not they the same thing?
Thanks for any hint
Regards,

Qatan

[Przemyslaw Czerpak]

On Mon, 17 Sep 2012, Qatan wrote:

Hi,

> >You can use DBFFPT, DBFCDX, SIXCDX, DBFNSX and they will work without any
> >problems.
> I just would like to understand a bit what is the difference
> between DBFNSX and SIXCDX, please. Are not they the same thing?

SIXCDX in Harbour is slightly modified DBFCDX.
SIX3 CDX implementation has many different bugs which can be exploited
quite fast, i.e. prefix packing of trailing spaces in CDX leaf nodes
causes that Clipper SIX3 applications GPFs due to buffer overflow when
they read such indexes. So I created SIXCDX for SIX3 users who want to
read CDX files created by Harbour and disabled in it some optimizations
which I know can exploit bugs in SIX3 CDX implementation. So the main
difference between DBFCDX and SIXCDX in Harbour is the fact that the
second one is a little bit less efficient.
There are also some other differences but they are really minor, i.e.
the order of calling EVAL/WHILE/FOR blocks in indexing, etc.
DBFCDX in Harbour is by default Clipper 5.3 DBFCDX / COMIX compatible
in such places and SIXCDX tries to replicate Clipper 5.2 DBFCDX / Six3
behavior. In 99% of cases such differences are not visible for users.

best regards,
Przemek

[Qatan]

Hello Przemek,

Well, if I understood well the best is to use DBFCDX with COMIX LOCK
SCHEMA.
But you didn't mention anything about DBFNSX (is the same as DBFCDX with
only the extension name being different?) So does not make any difference to
use DBFCDX or DBFNSX?

What I am doing right now is:

------8<------
request DBFNSX
request DBFSMT
...
RDDSETDEFAULT( 'DBFNSX' )
RDDREGISTER( 'DBFNSX', RDT_FULL )
RDDINFO( RDDI_MEMOTYPE, DB_MEMO_SMT )

SET AUTOPEN ON // automaticaly opens the index file (with the same
name)
SET AUTORDER TO 1 // automaticaly selects the first order of an opened
index file
SET OPTIMIZE ON
SET HARDCOMMIT ON
SET EXCLUSIVE OFF
SET AUTOSHARE TO 1 // opens database shared in a network and exclusive if
no network is detected
SET MEMOBLOCK TO 16
SET DBFLOCKSCHEME TO DB_DBFLOCK_COMIX // COMIX and CL53 DBFCDX hyper
locking scheme
// use COMIX like hyper locking mode - it should increase performance
------>8------

Is that good or I should improve?
Just to let you understand my need (and I believe maybe most of the
actual users) - I do not need (or care about) compatibility with Clipper or
any lib - I just want to use the best available way. One thing that attracts
me is the possibility of using cryptographed database so I am not sure I can
do it using DBFNSX (or DBFCDX) and sx_dbfencrypt() (if I remember right) or
I have to use SIXCDX that seems less efficient to be able to encrypt the
database? I believe Harbour implementation is way superior to any possible
Clipper one (even if it is fully compatible).
Sorry, I may look confused (and I am...) and I never used any DBF database
with Harbour (only a bit with Clipper/Six3)

One thing I notice is that seems some functions only have effect when
using SIXCDX (those sx_*()). Is that right?

Again... thanks for your patience.

Qatan

[Przemyslaw Czerpak]

On Mon, 17 Sep 2012, Qatan wrote:

Hi,

> >You can use DBFFPT, DBFCDX, SIXCDX, DBFNSX and they will work without any
> >problems.
> I just would like to understand a bit what is the difference
> between DBFNSX and SIXCDX, please. Are not they the same thing?

I descibed it few times in the past.
Lets forget about SIXCDX - it's slighlt modified DBFCDX not worth to
talk about.

DBT, FMT, SPT are different MEMO formats. All of them are supported
by Harbour and automatically recognized when DBF file is open.
NTX, CDX and NSX are different index formats. They can be used
in any combinations of MEMO formats, i.e. DBFCDX perfectly well
works with DBT memo files just like with FPT and SMT ones.

In Harbour all core RDDs using above index formats (DBFNSX, DBFCDX, DBFNSX)
have nearly the same functionality which cover nearly all index feautres
known in xbase world and many of them are unique to [x]Harbour only so they
are not supported by other drivers.
With all above RDDs user can use all ord*(), db*(), sx_*(), hsx_*() , ...
functions, can create multitag indexes (many orders in single file, also
for NTX format), autooreder, autoopen, production indexes, etc. so for
programmer used RDD should not create any difference. It's also possible
to enable disable some features using RDDI_* interface, i.e. this code
change default DBFNTX behavior so it behaves just like DBFCDX and even
uses ".cdx" as default file extnesion (of course internally it's still
NTX format with Harbour extenssions - we support CTX format from CLIP)
// default index extenssion
rddInfo( RDDI_ORDBAGEXT, ".cdx", "DBFNTX" )
//support multi tag in single index file
rddInfo( RDDI_MULTITAG , .t. , "DBFNTX" )
// structural indexes support
rddInfo( RDDI_STRUCTORD, .t. , "DBFNTX" )
// record number is hidden trailing part of key duirng sorted
rddInfo( RDDI_SORTRECNO, .t. , "DBFNTX" )

There are only few minor excpetions rather unimportant for most of users
unique to given RDDs. The most important are two:
1. I implemented dynamic unique indexes only in DBFCDX
It means that ordUnique( ,, .t. ) -> can enable/disable unique
mode only in DBFCDX (and SIXCDX)
2. Only in DBFNTX and DBFNSX I implement special mode which allows to
use page numbers instead of file offsets in index pages. In this
mode indexes are not binary compatible with other languages but
their maximal size has been greatly extended and for NTX and NSX
files is 2^32 * index_page_size what gives 2^42 for default 1024
pages in this formats - 2^42 is 4TB
I haven't implemented it in DBFCDX so far and for this format
maximum index size is still 4GB. Maybe in some spare time I'll
do that and also add support for different page sizes.
BTW in ADS .adi indexes are slightly modified CDX files where
page numbers are used instead of offsets and index page size
can be changed.

Of course there are very serious differences in low level implementation
and structures used by these formats.
NTX is simple BTREE without any compression. The operation are extremely
fast but indexes are much bigger then in CDX or NSX format so performance
is usually storngly reduced by cost of IO operations. Anyhow theoretically
having very strong server application with a lot of RAM so all data are
accessed from memory not from harddisks this is the best choice.
CDX and NSX compress leaf nodes so total size of index files is much
smaller then in NTX format. NSX uses simple BTREE when CDX uses three
of the most significant keys. It means that update in CDX files can
be more expensive then in NSX format especially if we are adding keys
which should be sorted as last (it's the most common situation, i.e.
when we are adding records with current date) because all nodes from
leaf to root have to be updated. All keys are repeated in leaf nodes
and there are internal bindings for nodes on the same level (BTW some
RDDs like SIX3 do not update them correctly for interior nodes) so CDX
format is also a little bit redundant. Anyhow all keys are repeated
in leafs nodes and we have internal bindings between all leaves so
skipping can be a little bit faster, etc.
Page size in CDX is smaller then in NSX so for very long keys, i.e.
over 100 bytes NSX format should be much more efficient. It also uses
different compression method which should be better for keys having
long space substrings inside, i.e. due to concatenation of few longer
fields like: FNAME[40] + LNAME[40]
In general this is to big subject to describe it here in few words.
In include/hbrddnsx.h I've made some small description of NSX format
when I was implementing it.

best regards,
Przemek

[Qatan]

Hello Przemek,

>I descibed it few times in the past.

Thanks for your explanation about indexes (again...)
So the way I am planning to setup it seems to be the best, right?
For filter optimization so I should use ordscope?
RDDInfo() and DBInfo() are both very interesting. I will study about
them.
Thanks again

------8<------
request DBFNSX
request DBFSMT
...
RDDSETDEFAULT( 'DBFNSX' )
RDDREGISTER( 'DBFNSX', RDT_FULL )
RDDINFO( RDDI_MEMOTYPE, DB_MEMO_SMT )

SET AUTOPEN ON // automaticaly opens the index file (with the same
name)
SET AUTORDER TO 1 // automaticaly selects the first order of an opened
index file
SET OPTIMIZE ON
SET HARDCOMMIT ON
SET EXCLUSIVE OFF
SET AUTOSHARE TO 1 // opens database shared in a network and exclusive if
no network is detected
SET MEMOBLOCK TO 16
SET DBFLOCKSCHEME TO DB_DBFLOCK_COMIX // COMIX and CL53 DBFCDX hyper
locking scheme
// use COMIX like hyper locking mode - it should increase performance
------>8------

Regards,

Qatan