hb_cdxflush

[shoves...@gmail.com]

My company is in the process of switching over from xHarbour to harbour. Recently we've had some issues with indexes becoming corrupt  (more so out of date on a few random records) and I was wondering if harbour falls prey to the same issue. 

*Speculation ahead* 

I suspect that the issue occurs because we lock then write the dbf then wait for a write lock on the cdx then write to the cdx, then unlock both. I think that users occasionally kill the process after the dbf write before the cdx write because of the many concurrent users slowing down the locking process.

I tracked your logic down to hb_cdxflush in dbfcdx_1, and it looks like it does the same thing. Is there a reason that the 

dbf lock(slow)

dbf write(fast)

cdx lock(slow)

cdx write(relatively fast)

both unlock(fast)

pattern is used rather than a 

dbf lock(slow)

cdx lock(slow)

cdx write(relatively fast)/dbf write(fast)

both unlock(fast)

pattern? 

I may be reading the code incorrectly, but I think the second pattern would prevent more issues. 

-Seth

[Aleksander Czajczynski]

Hi Seth!

shoves...@gmail.com wrote:
> My company is in the process of switching over from xHarbour to
> harbour. Recently we've had some issues with indexes becoming corrupt
> (more so out of date on a few random records) and I was wondering if
> harbour falls prey to the same issue.
>
> *Speculation ahead*
> I suspect that the issue occurs because we lock then write the dbf
> then wait for a write lock on the cdx then write to the cdx, then
> unlock both. I think that users occasionally kill the process after
> the dbf write before the cdx write because of the many concurrent
> users slowing down the locking process.

Would be good to prevent killing of the app. Don't know your setup, you
may:
- switch from GTWIN to GTWVT and prevent killing with [X]?
- use screen or sth like that if it's SSH session on an unix box
- examine other terminal solutions...

>
> I tracked your logic down to hb_cdxflush in dbfcdx_1, and it looks
> like it does the same thing. Is there a reason that the
> dbf lock(slow)
> dbf write(fast)
> cdx lock(slow)
> cdx write(relatively fast)
> both unlock(fast)

Flush order is not write order (in logical sense). I hope i'm not
messing things either.

Write pattern is more like this (as i see it):
- dbf lock (in 1:1 relation with .prg code)
- cdx lock (following actual changes to the record)
- cdx write
- cdx unlock
- dbf write
- (opt.) dbf flush
- (opt.) cdx flush
- dbf unlock

Flush pattern seems to be reverted to writes and with HARDCOMMIT set by
default,
this makes DBCommit() instruct OS to flush disk buffers.
Physical write is not abandoned by OS when process is killed.

Best regards, Aleksander

[testi testmann]

hi,

i guess you are talking about Network ?
so what OS is your Server and what OS is your Client ?

since Windows Vista we have SMB2 and a "local Cache" which make Problem and corrupt Data.

more Information read here
http://technet.microsoft.com/en-us/library/ff686200%28WS.10%29.aspx

greetings

Jimmy