Does pydbf do any file locking or multi-user contention handling?

108 views
Skip to first unread message

Mike Morris

unread,
Nov 19, 2014, 6:18:32 PM11/19/14
to python...@googlegroups.com
I've written a small tool to allow editing some Foxpro DOS 2.6 tables. Some columns have been added to the tables since the app was written, and the original FoxPro app does not support editing them. The pydbf tools provides write access to those fields, and works perfectly when it is the only app accessing the tables.

However, when using the pydbf tool while there are instances of the original application open, results are very inconsistent.

Various results in multi-user situations include:
  • the record is updated immediately (actually, within the SET REFRESH() time of the original app)
  • it updates when the original app *on the same workstation as the pydbf tool is running* is closed
  • it updates only when all instances of the original app are closed
  • it never updates at all

I have not conducted rigorous testing, so I can't really *swear* by all those results, but it's definitely inconsistent, and presumably dependent upon the lock state of the table/record in running instances of the old app... but I'm hoping I can figure it out by just understanding any multi-user related pydbf behavior :-)

Thanks in advance - and thanks for the pydbf tool, it's great!

Ethan Furman

unread,
Nov 19, 2014, 6:31:13 PM11/19/14
to python...@googlegroups.com
On 11/19/2014 03:18 PM, Mike Morris wrote:

[snip]

I think you have the wrong group -- pydbf is a different dbf library than python-dBase (or simply dbf).

I can say that my library is not multi-user friendly as far as contentions go, but it does take pains to only read or
write whichever record is being used. Still, I would not recommend it for multi-user settings.

--
~Ethan~

signature.asc

Adrian Klaver

unread,
Nov 19, 2014, 6:35:36 PM11/19/14
to python...@googlegroups.com
On 11/19/2014 03:18 PM, Mike Morris wrote:
> I've written a small tool to allow editing some Foxpro DOS 2.6 tables.
> Some columns have been added to the tables since the app was written,
> and the original FoxPro app does not support editing them. The pydbf
> tools provides write access to those fields, and works perfectly when it
> is the only app accessing the tables.
>
> However, when using the pydbf tool while there are instances of the
> original application open, results are very inconsistent.
>
> Various results in multi-user situations include:
>
> * the record is updated immediately (actually, within the SET
> REFRESH() time of the original app)
> * it updates when the original app *on the same workstation as the
> pydbf tool is running* is closed
> * it updates only when all instances of the original app are closed
> * it never updates at all
>
> I have not conducted rigorous testing, so I can't really *swear* by all
> those results, but it's definitely inconsistent, and presumably
> dependent upon the lock state of the table/record in running instances
> of the old app... but I'm hoping I can figure it out by just
> understanding any multi-user related pydbf behavior :-)

My experience is that dbf only really works for multi-users when they
are logged into and accessing through a single application. My worse
case experience is corrupted dbf files.

>
> Thanks in advance - and thanks for the pydbf tool, it's great!
>
> --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "Python dBase" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to python-dbase...@googlegroups.com
> <mailto:python-dbase...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
Adrian Klaver
adrian...@aklaver.com

Mike Morris

unread,
Nov 19, 2014, 8:22:30 PM11/19/14
to python...@googlegroups.com


On Wednesday, November 19, 2014 3:31:13 PM UTC-8, Ethan Furman wrote:

I think you have the wrong group -- pydbf is a different dbf library than python-dBase (or simply dbf).

Hmmm... I see I'm confused :-) Thanks for your gracious patience with me!

But I think I just used the wrong name... I've installed the package named "python-dbf" from ubuntu repos, which has this group linked as the home page. I don't know where I got the "pydbf" moniker... so I'm pretty sure I'm actually using your library.

Thanks for the answer, multi-user not recommended. While I'm here I have to ask... do you know of any python libraries that can handle a multi-user environment?

Thanks again,

MikeM

Ethan Furman

unread,
Nov 19, 2014, 11:15:32 PM11/19/14
to python...@googlegroups.com
On 11/19/2014 05:22 PM, Mike Morris wrote:
> On Wednesday, November 19, 2014 3:31:13 PM UTC-8, Ethan Furman wrote:
>>
>> I think you have the wrong group -- pydbf is a different dbf library than python-dBase (or simply dbf).
>
> But I think I just used the wrong name... I've installed the package named "python-dbf" from ubuntu repos, which has
> this group linked as the home page. I don't know where I got the "pydbf" moniker... so I'm pretty sure I'm actually
> using your library.

In that case, welcome! :)


> Thanks for the answer, multi-user not recommended. While I'm here I have to ask... do you know of any python libraries
> that can handle a multi-user environment?

As far as dbf file processing goes, mine is the best -- I wrote it because none of the others supported the features I
needed. (To be fair, I haven't really looked at the field in the last five years, so maybe somebody else has written a
better one than mine in the mean-time.)

Having said that, there is one possibility -- there is Python code associated with GIS, which has its own dbf library.
I haven't look closely at it (IIRC it's a GNU license, whilst I prefer the MIT/BSD), so maybe it has multi-user support.

Good luck!

--
~Ethan~

signature.asc

Adrian Klaver

unread,
Nov 19, 2014, 11:18:55 PM11/19/14
to python...@googlegroups.com
On 11/19/2014 05:22 PM, Mike Morris wrote:
>
>
From what I have seen it is not the library it is the OS. If you are
working with dbf files directly from different applications you are
depending on the OS to handle the locking. It has no understanding of
what the various applications are doing internally to the files. You
would get it the same situation if you worked directly with the database
files directly of a database like Postgres, rather then through the server.

>
> Thanks again,
>
> MikeM

Ethan Furman

unread,
Nov 19, 2014, 11:30:28 PM11/19/14
to python...@googlegroups.com
On 11/19/2014 08:18 PM, Adrian Klaver wrote:
> On 11/19/2014 05:22 PM, Mike Morris wrote:
>>
>> Thanks for the answer, multi-user not recommended. While I'm here I have
>> to ask... do you know of any python libraries that can handle a
>> multi-user environment?
>
> From what I have seen it is not the library it is the OS. If you are working with dbf files directly from different
> applications you are depending on the OS to handle the locking. It has no understanding of what the various applications
> are doing internally to the files. You would get it the same situation if you worked directly with the database files
> directly of a database like Postgres, rather then through the server.

I believe the later dbf formats have rudimentary (and possible better) support for locking built-in -- but I didn't need
it, and it looked to be a pain to implement, so I didn't bother. (Note that dbf's support only goes to dBase III and
VFP6 (and Clipper) -- dBase IV+ and VFP7+ may have good support for it, I don't know.)

--
~Ethan~

signature.asc

Adrian Klaver

unread,
Nov 20, 2014, 10:12:10 AM11/20/14
to python...@googlegroups.com
I did a Google search on "multi user locking dbf files" and the
information is all over the map. Best case seems to be 'dirty' read
where a record is locked for write, but can be read. Being able to do
this seems to be dependent on where the files are(local/network drive),
what OS(&version) and the applications involved. All in all not an easy
task.

>
> --
> ~Ethan~
>


--
Adrian Klaver
adrian...@aklaver.com
Reply all
Reply to author
Forward
0 new messages