Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Slow DBF shared access over the LAN

2,633 views
Skip to first unread message

Enrico Maria Giordano

unread,
Sep 22, 2012, 5:02:19 PM9/22/12
to
As per request, this is a new thread dedicated to the problem of the slow
DBF shared access over the LAN.

In the past we had Clipper and very slow PCs and LANs (if compared with the
current ones). Despite of that, our application were very fast. Now that we
have powerful PCs and faster LANs the shared acces to DBF is slower,
somewhat unacceptably slower.

Any ideas? Anything that can be done at DBF driver level?

EMG

--
EMAG Software Homepage: http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page: http://www.emagsoftware.it/emgmusic


fatfat

unread,
Sep 23, 2012, 5:01:26 AM9/23/12
to
we have the same problem with 2008 server + Win 7 Client, but Win 7 (shared)
+ Win 7 Client got acceptable speed : )

Thanks


"Enrico Maria Giordano" 在郵件張貼內容主旨
505e275a$0$13277$4faf...@reader2.news.tin.it 中撰寫...

Enrico Maria Giordano

unread,
Sep 23, 2012, 5:38:05 AM9/23/12
to
> we have the same problem with 2008 server + Win 7 Client, but Win 7
> (shared)
> + Win 7 Client got acceptable speed : )

So it looks like a Windows configuration problem...

Patrick Mast

unread,
Sep 23, 2012, 6:15:23 AM9/23/12
to
Hello,

> we have the same problem with 2008 server + Win 7 Client, but Win 7 (shared)
> + Win 7 Client got acceptable speed : )

Still... The ONE main problem we have with our app is slow network access of DBF files. Yes, we also tried TS and lately SQLRDD. But xHarbour should be able to give us acceptable speed in a Windows network.

My 2cents

Patrick

Douglas Woodrow

unread,
Sep 23, 2012, 8:32:34 AM9/23/12
to
On Sat, 22 Sep 2012 23:02:19, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>As per request, this is a new thread dedicated to the problem of the slow
>DBF shared access over the LAN.
>
>In the past we had Clipper and very slow PCs and LANs (if compared with the
>current ones). Despite of that, our application were very fast. Now that we
>have powerful PCs and faster LANs the shared acces to DBF is slower,
>somewhat unacceptably slower.
>I know that we can use client/server or terminal server solutions but
>the legacy application using DBF would really need a speed boost...
>
>Any ideas? Anything that can be done at DBF driver level?

IMO the best solution is the Advantage Database Server, which supports
existing DBF-based databases as well as their proprietary (ADT) database
format, and is very fast (particularly noticeable when there are lots of
users).

There are Advantage RDD drivers for both Clipper and Harbour so it is a
trivial matter to convert existing applications to work with ADS, and
you can then take advantage of SQL access to the same database.

Of course this isn't a free solution, but if your clients are
complaining about slow access over the LAN they should be prepared to
pay for it, and it's very easy to implement.

The Advantage driver does also support free "local" access to the tables
(i.e. not using the database server), I'm not sure how the speed
compares to the other Harbour DBF RDDs.

--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 23, 2012, 8:47:01 AM9/23/12
to
> IMO the best solution is the Advantage Database Server,

Too expensive, I'm afraid. And beside this, what about complex filters
containing function calls and other fancy things? As far as I know, ADS
doesn't support it. Or am I wrong?

fatfat

unread,
Sep 23, 2012, 8:49:04 AM9/23/12
to
already apply the same reg that need for file sharing with NT Server + DBF
on both case.


"Enrico Maria Giordano" 在郵件張貼內容主旨
505ed87e$0$13282$4faf...@reader2.news.tin.it 中撰寫...

Enrico Maria Giordano

unread,
Sep 23, 2012, 8:50:23 AM9/23/12
to
And what about tests for the existence of DBF files, index etc.? My concern
is that I can't just take my source code, recompile it and voilą, I'm
afraid. If this is true, I can't take advantage of the... Advantage. :-)

fatfat

unread,
Sep 23, 2012, 8:50:52 AM9/23/12
to
already try ADS / local access, but it seems can't co-working with rddcdx

"Douglas Woodrow" 在郵件張貼內容主旨
kg48HDCi...@woodrowhorsfall.plus.com 中撰寫...

Douglas Woodrow

unread,
Sep 23, 2012, 9:34:58 AM9/23/12
to
On Sun, 23 Sep 2012 20:50:52, fatfat <fat...@fatfat.com> wrote
>already try ADS / local access, but it seems can't co-working with rddcdx

Obviously writing to the same tables via different mechanisms is never a
very good idea, but what exactly was the problem? Did you try changing
the Advantage locking method from "proprietary" to "compatible"?
--
Doug Woodrow

Douglas Woodrow

unread,
Sep 23, 2012, 9:31:12 AM9/23/12
to
On Sun, 23 Sep 2012 14:50:23, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>And what about tests for the existence of DBF files, index etc.?

I'm not sure what you're getting at, ADS doesn't stop you making system
calls to check the existence of files. It does have a security option
to enable it to open the database even when the logged-in user of the
client application does not have network access to the files, maybe
that's what you're referring to?

>My concern
>is that I can't just take my source code, recompile it and voilą, I'm
>afraid. If this is true, I can't take advantage of the... Advantage. :-)

There may be some areas that need some attention, but in my experience
it was very easy.
--
Doug Woodrow

Douglas Woodrow

unread,
Sep 23, 2012, 9:23:43 AM9/23/12
to
On Sun, 23 Sep 2012 14:47:01, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote

>what about complex filters
>containing function calls and other fancy things? As far as I know, ADS
>doesn't support it. Or am I wrong?

AFAIK ADS does support filters containing function calls, but then of
course the filtering has to happen at the client end, so you lose much
of the benefit of using a client-server configuration. It is much
better to get the server to do the filtering, so only a small number of
data packets are sent across the network. I suspect any of your
"complex" filters could be re-written in SQL so that the filtering would
be performed by the server.

--
Doug Woodrow

fatfat

unread,
Sep 23, 2012, 10:02:49 AM9/23/12
to
may be it is the case of locking method from ADS : (
b'cos I have modified to use AIS but got update problem together with
rddcdx.



"Douglas Woodrow" 在郵件張貼內容主旨
Ngg2HxEC...@woodrowhorsfall.plus.com 中撰寫...

Enrico Maria Giordano

unread,
Sep 23, 2012, 10:10:03 AM9/23/12
to
> AFAIK ADS does support filters containing function calls, but then of
> course the filtering has to happen at the client end, so you lose much of
> the benefit of using a client-server configuration. It is much better to
> get the server to do the filtering, so only a small number of data packets
> are sent across the network. I suspect any of your "complex" filters
> could be re-written in SQL so that the filtering would be performed by the
> server.

You missed the point. I'm talking about big legacy applications that would
require too much work to adapt to the SQL way.

Enrico Maria Giordano

unread,
Sep 23, 2012, 10:14:17 AM9/23/12
to
> I'm not sure what you're getting at, ADS doesn't stop you making system
> calls to check the existence of files. It does have a security option to
> enable it to open the database even when the logged-in user of the client
> application does not have network access to the files, maybe that's what
> you're referring to?

No, just something like

IF !FILE( "MYTABLE.DBF" )
// Create it
ENDIF

> >My concern
> >is that I can't just take my source code, recompile it and voilą, I'm
> >afraid. If this is true, I can't take advantage of the... Advantage. :-)
>
> There may be some areas that need some attention, but in my experience it
> was very easy.

I'd love to do a test with my applications. Any guides? Where do I have to
start? Downloading something, I suppose.

Douglas Woodrow

unread,
Sep 23, 2012, 1:19:09 PM9/23/12
to
On Sun, 23 Sep 2012 16:14:17, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>> I'm not sure what you're getting at, ADS doesn't stop you making system
>> calls to check the existence of files. It does have a security option to
>> enable it to open the database even when the logged-in user of the client
>> application does not have network access to the files, maybe that's what
>> you're referring to?
>
>No, just something like
>
>IF !FILE( "MYTABLE.DBF" )
> // Create it
>ENDIF

That's a simple operating system call, nothing to do with ADS, you can
still do that.


>I'd love to do a test with my applications. Any guides? Where do I have to
>start? Downloading something, I suppose.

Yes, download a server (either a trial version or I think developers can
still obtain free 2-user versions), and build the Harbour rddads library
for your appropriate version.

The Advantage guys (as part of Sybase) have recently moved (regrettably
IMO) to a web-based forum:
http://devzone.advantagedatabase.com/forum

although their newsgroups are still available in read-only form online:
http://devzone.advantagedatabase.com/dz/content.aspx?key=7

--
Doug Woodrow

Douglas Woodrow

unread,
Sep 23, 2012, 1:25:42 PM9/23/12
to
On Sun, 23 Sep 2012 16:10:03, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>> AFAIK ADS does support filters containing function calls, but then of
>> course the filtering has to happen at the client end, so you lose much of
>> the benefit of using a client-server configuration.

>You missed the point. I'm talking about big legacy applications that would
>require too much work to adapt to the SQL way.

Sorry if I wasn't clear.

(1) Yes, ADS _does_ support filter expressions that have to be evaluated
on the client (and it has an intelligent engine that will try to
evaluate the filter on the server if possible). Thus without any change
to your application you would gain the client-server speed benefit of
having a dedicated server handling reading and writing to your data
files, as opposed to multiple clients each making individual file access
requests over the network.

(2) (referring to the main point of this thread, that of slow data
access over a LAN) Once you are using a client-server solution, you can
then update your application to improve the speed of the slow filters by
moving the evaluation from the client to the server.
--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 23, 2012, 5:23:49 PM9/23/12
to
> That's a simple operating system call, nothing to do with ADS, you can
> still do that.

Good.

> Yes, download a server (either a trial version or I think developers can
> still obtain free 2-user versions), and build the Harbour rddads library
> for your appropriate version.

Thank you, I will try.

Enrico Maria Giordano

unread,
Sep 23, 2012, 5:26:03 PM9/23/12
to
> Sorry if I wasn't clear.
>
> (1) Yes, ADS _does_ support filter expressions that have to be evaluated
> on the client (and it has an intelligent engine that will try to evaluate
> the filter on the server if possible). Thus without any change to your
> application you would gain the client-server speed benefit of having a
> dedicated server handling reading and writing to your data files, as
> opposed to multiple clients each making individual file access requests
> over the network.
>
> (2) (referring to the main point of this thread, that of slow data access
> over a LAN) Once you are using a client-server solution, you can then
> update your application to improve the speed of the slow filters by moving
> the evaluation from the client to the server.

Very good, thank you for the info.

dlzc

unread,
Sep 23, 2012, 5:41:46 PM9/23/12
to
Dear Enrico Maria Giordano:

On Saturday, September 22, 2012 2:02:21 PM UTC-7, Enrico Maria Giordano wrote:
...
> As per request, this is a new thread dedicated
> to the problem of the slow DBF shared access
> over the LAN.

In general, is this a problem only in backwards compatibility (server 2008 to Win7), peer-to-peer, or mixed?

> In the past we had Clipper and very slow PCs
> and LANs (if compared with the current ones).
> Despite of that, our application were very
> fast. Now that we have powerful PCs and faster
> LANs the shared acces to DBF is slower, somewhat
> unacceptably slower.

I find this endemic on newer user OS, with older server OS. For example:
http://bytes.com/topic/access/answers/208767-access-really-slow-over-network

And then there is this perennial problem...
http://support.microsoft.com/kb/296264

> Any ideas? Anything that can be done at DBF
> driver level?

You have been directed by others to commercial server-side solutions.

Options might be a roll-your-own type solution, perhaps server commands via direct network packet exchange, transmitting xbscript commands, receiving datafile subsets, processing input batches, ... whatever.

David A. Smith

Enrico Maria Giordano

unread,
Sep 23, 2012, 6:06:10 PM9/23/12
to
> In general, is this a problem only in backwards compatibility
> (server 2008 to Win7), peer-to-peer, or mixed?

Mixed, I would say. But I'm not really sure.

> Options might be a roll-your-own type solution, perhaps server
> commands via direct network packet exchange, transmitting
> xbscript commands, receiving datafile subsets, processing
> input batches, ... whatever.

Whatever solution we think has to be transparent to the application or it
will be unuseful.

Enrico Maria Giordano

unread,
Sep 23, 2012, 6:12:49 PM9/23/12
to
Is this the correct file to install?

http://devzone.advantagedatabase.com/dz/content.aspx?key=20&Release=18&Product=25&Platform=12

I get lost in the Advantage website...

Jayadev

unread,
Sep 24, 2012, 1:55:50 AM9/24/12
to
Dear Enrico,

I have successfully used OTC Mediator+xHarbour+BCC 5.8.2+MySql. No change in
program syntax, substantial improvement in speed and you can use sql
statements also.

Mediator is free for MySql (100 users) and for MS SQL it is free for 3 users
I believe.

site address for Mediator http://www.otc.pl/download/default.aspx?l=2

HTH,

Warm regards,

Jayadev


"Enrico Maria Giordano" <e.m.gi...@emagsoftware.it> wrote in message
news:505e275a$0$13277$4faf...@reader2.news.tin.it...

Enrico Maria Giordano

unread,
Sep 24, 2012, 4:37:06 AM9/24/12
to
> I have successfully used OTC Mediator+xHarbour+BCC 5.8.2+MySql. No change
> in program syntax, substantial improvement in speed and you can use sql
> statements also.
>
> Mediator is free for MySql (100 users) and for MS SQL it is free for 3
> users I believe.
>
> site address for Mediator http://www.otc.pl/download/default.aspx?l=2

Thank you, I will try it also. Any step by step instructions? :-)

Jayadev

unread,
Sep 24, 2012, 6:50:32 AM9/24/12
to
Dear Enrico,

> 1. Install the Mediator server software on the server/machine where your
> MySql installation is present.

>>What about if MySQL is on a server that is not mine >>and where I cannot
>>install anything (the most frequently case for me)?

You only have to install the Mediator server once and create a system DSN on
server.

>>Additionaly, what about DBCreate() function, >>backup/restore DBFs and
>>test for the existence of DBF files?

All the above queires are handled with the same commands as in
clipper/xharbour, pls go thru the manual available with the product. Backup
can be thru MySqlDump.exe

>>It looks like mediator is not useful for me,
Try it, you will find it to be a very easy and quick way for you to convert
from dbfs to sql.
All queries are executed at the server and complex filters can be set on SQL
DB giving considerable speed in queries using 'select where orderby etc'.
We can get the results in dbf format and use the resultant table with 'use
command'.

HTH,

Warm regards,

Jayadev



"Enrico Maria Giordano" <e.m.gi...@emagsoftware.it> wrote in message
news:505f8977$0$13281$4faf...@reader2.news.tin.it...

Patrick Mast

unread,
Sep 24, 2012, 8:21:27 AM9/24/12
to
Guys,

Advantage, Mediator, SQLRDD, all these client/server solutions work but..
Should we not have FAST database access via our core DBF engines?

The fix is not to look away for 3rd party solutions but to look inside and fix what we have.

my 2 cents ;-)

Patrick

Enrico Maria Giordano

unread,
Sep 24, 2012, 9:11:45 AM9/24/12
to
> > What about if MySQL is on a server that is not mine
> > and where I cannot install anything (the most frequently
> > case for me)?
>
> You only have to install the Mediator server once and create
> a system DSN on server.

What I cannot do, sorry.

> > Additionaly, what about DBCreate() function, backup/restore
> > DBFs and test for the existence of DBF files?
>
> All the above queires are handled with the same commands as
> in clipper/xharbour,

I can't see how. SQL database is a different thing than DBFs. If now I have,
for instance, MYTABLE.DBF, I can save it and check for its existance using
FILE( "MYTABLE.DBF" ). Does it work for a SQL table too?

> pls go thru the manual available with the product. Backup
> can be thru MySqlDump.exe

The point of this thread is: how get more speed from the legacy applications
using DBFs over a LAN? I can't see how Mediator can help in this subject. I
hope to be wrong.

> Try it, you will find it to be a very easy and quick way for
> you to convert from dbfs to sql.

This is the point: I don't want to convert anything. I already use SQL
engines through ADO with full satisfaction.

Enrico Maria Giordano

unread,
Sep 24, 2012, 9:12:42 AM9/24/12
to
> Advantage, Mediator, SQLRDD, all these client/server solutions work but..
> Should we not have FAST database access via our core DBF engines?
>
> The fix is not to look away for 3rd party solutions but to look inside and
> fix what we have.

I totally agree!

Douglas Woodrow

unread,
Sep 24, 2012, 9:19:44 AM9/24/12
to
On Mon, 24 Sep 2012 00:12:49, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>Is this the correct file to install?
>
>http://devzone.advantagedatabase.com/dz/content.aspx?key=20&Release=18&Product=
>25&Platform=12


Hi Enrico,

Version 11 is the most recent release, it should be fine although I
believe the latest Harbour version includes the rddads library for
version 10.1 ready built so that might be easier for you initially:

http://devzone.advantagedatabase.com/dz/content.aspx?key=20&Release=16

I think you'll need "Advantage Database Server" and "Advantage Client
Engine", and "Advantage Data Architect" is very useful too.

Here is a simple self-contained example to show you how easy it is to
incorporate in your existing application:

//-----------------------------------------------------------------
#include "ADS.ch"

REQUEST ADS

Procedure Main( cDataPath )
If cDataPath == NIL
cDataPath := "\\WinServer\TESTdata\"
End
? "DB connection path:", cDataPath
InitADS( cDataPath )

// simple example of using SQL to fetch the time on the ADS server
If RunSQL( "SELECT now() AS [ServerTime] FROM SYSTEM.iota")
? "ADS server time:", _TEMPQRY->ServerTime
Endif

If CreateTestTable()
USE (cDataPath + "TestTable") ALIAS TEMP SHARED NEW
? "Number of records in TestTable:", TEMP->(LastRec())
TEMP->(OrdSetFocus("FIELD1"))
If TEMP->(DbSeek(2))
? "Record 2 found"
Else
? "ERROR: Record 2 not found!"
Endif
? "All records:"
TEMP->(DbEval( {|| QOut(TEMP->Field1, TEMP->Field2) }))
TEMP->( DbSetFilter( {|| MyFilterFunction() }, "Local filter"))
? "Filtered:"
TEMP->(DbEval( {|| QOut(TEMP->Field1, TEMP->Field2) }))
Endif
Return

Static Procedure InitADS( cPath )
RddSetDefault("ADS")
//AdsSetServerType( ADS_LOCAL_SERVER + ADS_REMOTE_SERVER )
AdsSetServerType( ADS_REMOTE_SERVER )
AdsSetFileType( ADS_CDX )
AdsConnect( cPath )
Return

Static Function RunSQL(cSQL, cWorkArea, TableType)
Local lRet := .F.
Local nError
Local cError

If cWorkArea == NIL
cWorkArea := "_TEMPQRY"
End
If TableType == NIL
TableType := AdsSetFileType()
Else
AdsSetFileType( TableType )
End

DbSelectArea(Select(cWorkArea))
DbCloseArea()

If AdsCreateSQLStatement( cWorkArea, TableType )
lRet := ADSExecuteSQLDirect( cSQL )
End
If !lRet
nError := AdsGetLastError(@cError)
Alert ("ADS error #" + LTrim(Str(nError)) + ": " + cError)
End
Return( lRet )

Static Function CreateTestTable()
// You can still use DbCreate() etc., but SQL is easy:
Static cSQL := "TRY DROP TABLE TestTable; CATCH ALL END TRY;";
+ "CREATE TABLE TestTable (Field1 Numeric(3,0), Field2 CHAR(10));";
+ "CREATE INDEX FIELD1 ON TestTable (Field1);";
+ "INSERT INTO TestTable (Field1, Field2) VALUES (1, 'Primo');";
+ "INSERT INTO TestTable (Field1, Field2) VALUES (2, 'Secondo');";
+ "INSERT INTO TestTable (Field1, Field2) VALUES (3, 'Terzo');";
+ "INSERT INTO TestTable (Field1, Field2) VALUES (4, 'Quarto');"
Local lOk

lOk := RunSQL( cSQL )
Return( lOk )

Static Function MyFilterFunction()
Local lPassFilter := .F.
if 'e' $ FIELD->Field2
lPassFilter := .T.
endif
Return( lPassFilter )
//-----------------------------------------------------------------
--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 24, 2012, 9:38:13 AM9/24/12
to
Let's start with the simpler thing:

FUNCTION MAIN()

LOCAL xVal

USE MYTABLE SHARED

WHILE !EOF()
xVal = FIELD -> myfield
SKIP
ENDDO

RETURN NIL

Will the above code get any benefits in using ADS?

Douglas Woodrow

unread,
Sep 24, 2012, 9:56:53 AM9/24/12
to
On Mon, 24 Sep 2012 15:38:13, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>Let's start with the simpler thing:
>
>FUNCTION MAIN()
>
> LOCAL xVal
>
> USE MYTABLE SHARED
>
> WHILE !EOF()
> xVal = FIELD -> myfield
> SKIP
> ENDDO
>
> RETURN NIL
>
>Will the above code get any benefits in using ADS?

Yes, definitely, and especially when there is a lot of multi-user
activity on that table.

Why don't you try it?

You just need to specify the path to the data files, like so:
//-----------------------------------------------------------------
#include "ADS.ch"

REQUEST ADS

Procedure Main( cDataPath )
LOCAL xVal

? "DB connection path:", cDataPath
InitADS( cDataPath )

USE (cDataPath + "MYTABLE") SHARED

WHILE !EOF()
xVal = FIELD -> myfield
SKIP
ENDDO

Return

Static Procedure InitADS( cPath )
RddSetDefault("ADS")
AdsSetServerType( ADS_REMOTE_SERVER )
AdsSetFileType( ADS_CDX )
AdsConnect( cPath )
Return
//-----------------------------------------------------------------

--
Doug Woodrow

dlzc

unread,
Sep 24, 2012, 10:06:00 AM9/24/12
to
Dear Patrick Mast:

On Monday, September 24, 2012 5:21:27 AM UTC-7, Patrick Mast wrote:
...
> Should we not have FAST database access via
> our core DBF engines?

That is what this thread is supposed to be about.

> The fix is not to look away for 3rd party
> solutions but to look inside and fix what
> we have.

A third party solution *might* meet the constraints of this thread, but not client server.

The problem is, we have no more definition on the problem than we did when Elvira was sniping.

We have so far Win7 to Win7, no problem. Server 2008 to Win7, problem. No one knows what to fix, if we cannot narrow down where the problem is. Is it opportunistic locks, or something else?

> my 2 cents ;-)

Would help if we knew where breaks occurred, rather than start yet a different bandwagon "something is broken". Unless someone thinks this is well known, and I missed something.

David A. Smith

Enrico Maria Giordano

unread,
Sep 24, 2012, 10:18:51 AM9/24/12
to
> > Will the above code get any benefits in using ADS?
>
> Yes, definitely, and especially when there is a lot of multi-user
> activity on that table.

Great! :-)

> Why don't you try it?
>
> You just need to specify the path to the data files, like so:
> //-----------------------------------------------------------------
> #include "ADS.ch"
>
> REQUEST ADS
>
> Procedure Main( cDataPath )
> LOCAL xVal
>
> ? "DB connection path:", cDataPath
> InitADS( cDataPath )
>
> USE (cDataPath + "MYTABLE") SHARED

Wait a moment: are you asking me to change all the USE command of my
applications!!!

Enrico Maria Giordano

unread,
Sep 24, 2012, 10:22:49 AM9/24/12
to
> No one knows what to fix, if we cannot narrow down where the problem is.

You are right. Unfortunately, I have no way to make speed tests in various
configurations. Anybody could?

Douglas Woodrow

unread,
Sep 24, 2012, 12:08:39 PM9/24/12
to
On Mon, 24 Sep 2012 16:18:51, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>>
>> USE (cDataPath + "MYTABLE") SHARED
>
>Wait a moment: are you asking me to change all the USE command of my
>applications!!!

No, you can still run your application from the data directory if you
want to.

But I hope you're not telling me that you have USE commands dotted
around your production applications? Even in Clipper Summer'87 days it
was sensible to use a central function like Net_Use() for opening
tables. How can you deal consistently with network errors etc.
otherwise?
--
Doug Woodrow

dlzc

unread,
Sep 24, 2012, 12:16:24 PM9/24/12
to
Dear Enrico Maria Giordano:

On Monday, September 24, 2012 7:22:50 AM UTC-7, Enrico Maria Giordano wrote:
...
> > No one knows what to fix, if we cannot
> > narrow down where the problem is.
>
> You are right. Unfortunately, I have no way
> to make speed tests in various
> configurations.

I was not trying to buttonhole you, Enrico. You have identified one case, where speed suffers, and where it does not suffer. If others can identify other cases, then we can narrow the Universe down.

Anyone know of speed issues with Linux on either server, client, or both?

David A. Smith

Enrico Maria Giordano

unread,
Sep 24, 2012, 3:46:59 PM9/24/12
to
> No, you can still run your application from the data directory if you want
> to.

Ok.

> But I hope you're not telling me that you have USE commands dotted around
> your production applications? Even in Clipper Summer'87 days it was
> sensible to use a central function like Net_Use() for opening tables. How
> can you deal consistently with network errors etc. otherwise?

I cannot see how a generic opening function can solve the problem of
changing data path. You obviously cannot change it inside the generic
function because it wouldn't be generic anymore. Anyway, each programmer has
its personal way of organizing the code.

Enrico Maria Giordano

unread,
Sep 24, 2012, 3:53:00 PM9/24/12
to
> You have identified one case, where speed suffers, and where
> it does not suffer.

Unfortunately not. I'm only reporting here my customer's complains. But I
have absolutely no clear ideas about the problem, sorry... :-(

fatfat

unread,
Sep 25, 2012, 6:20:46 AM9/25/12
to
we have the same problem too, win7+win7 OK, win7+2008Server Slow on
filtering
I think rushmore like feature may help : )


"dlzc" 在郵件張貼內容主旨
029b85e9-9e10-4e11...@googlegroups.com 中撰寫...

fatfat

unread,
Sep 25, 2012, 6:21:56 AM9/25/12
to
need to have NETRDD : )


"Enrico Maria Giordano" 在郵件張貼內容主旨
50605c12$0$13281$4faf...@reader2.news.tin.it 中撰寫...

Otto Haldi

unread,
Sep 25, 2012, 6:57:39 AM9/25/12
to
On Mon, 24 Sep 2012 05:21:27 -0700 (PDT), Patrick Mast <patric...@xharbour.com> wrote:

Hello Patrick,

I agree. I'm reminder a long time ago, I moved a clipper 87 application from Novell to Win NT Server.
I lost about 20% of the speed to DBF databases access and never find a good solution with M$ OS.
A simple test is to scroll a table with DbBrowse.
At this time, I have head the same access time/speed as I used the Local HD or the Novell Network!

Regards
Otto

Pete

unread,
Sep 25, 2012, 7:08:03 AM9/25/12
to
Hi Enrico,

We too have problems with speed and also the occurence of the application
not responding message. Windows 7 does seem to be the common variable in
the installations we are having the problems.

We also noticed this problem happening when we upgraded our application -
which filters/scopes data in the same way but the changes made were
increasing the number of fields in the database & using Fivewins xBrowse
instead of the standard Fivewin browse. Running the system under SQL using
SQLRDD is ok and we dont notice the slow down or the not responding
messages.

We are currently investigating the common elements of the installations that
are having the issues and I will let you know if we find anything. We are
also testing the system by running it in windows xp compatibility mode to
see if this makes any difference.

Regards,

Pete

"Enrico Maria Giordano" <e.m.gi...@emagsoftware.it> wrote in message
news:505e275a$0$13277$4faf...@reader2.news.tin.it...
> As per request, this is a new thread dedicated to the problem of the slow
> DBF shared access over the LAN.
>
> In the past we had Clipper and very slow PCs and LANs (if compared with
> the current ones). Despite of that, our application were very fast. Now
> that we have powerful PCs and faster LANs the shared acces to DBF is
> slower, somewhat unacceptably slower.
>
> Any ideas? Anything that can be done at DBF driver level?
>

Ella Stern

unread,
Sep 25, 2012, 9:19:12 AM9/25/12
to
Hello,

> As per request, this is a new thread dedicated to the problem of the slow
>
> DBF shared access over the LAN.

In my point of view the problem's definition is the following: "How to reshape the communication system of a Windows 2008 server, in order to be optimal for a legacy application using file sharing instead of client-server data access?"

Here is an article about the SMB 2.1 protocol added to Windows 2008
http://technet.microsoft.com/en-us/library/ff625695(v=ws.10).aspx

One of the possible settings is enabling large MTU by using the appropiate registry settings.

RDDNTX, RDDCDX etc. are pure client-side modules, which enforce the server to send big data amounts over the wire for each and every client...but this is a different story...about software architectures.

Best regard,

Ella



dlzc

unread,
Sep 25, 2012, 10:20:57 AM9/25/12
to
Dear Ella Stern:

On Tuesday, September 25, 2012 6:19:12 AM UTC-7, Ella Stern wrote:
...
>
> > As per request, this is a new thread dedicated
> > to the problem of the slow DBF shared access
> > over the LAN.
>
> In my point of view the problem's definition is
> the following: "How to reshape the communication
> system of a Windows 2008 server, in order to be
> optimal for a legacy application using file
> sharing instead of client-server data access?"

Agree on all but choice of word "legacy". GUI-fication of an application will not change this behavior. And I know you meant "non-client-server" by that.

Here is a white paper from Micro$haft:
http://msdn.microsoft.com/en-us/windows/hardware/gg463394.aspx

Some configuration settings:
http://technet.microsoft.com/en-us/library/cc938576.aspx

And this fellow had success disabling NetDMA (with registry settings at need):
http://social.technet.microsoft.com/Forums/en/winserverfiles/thread/46898c7f-92e0-4c99-98d2-18a7458a7d2d

I think a new RDDNET might be a solution only if there is a server application. For just serving up data, or trying to consume data on the client side, no changes will improve speed.

David A. Smith

x

unread,
Sep 25, 2012, 11:58:35 AM9/25/12
to
dlzc,

Pritpal Bedhi was begin a very, very interesting Server-Client Rdd, but was
abandoned because poor harbourer's interest.
Too there is other named Leto Db (only harbour)

Regards.

"dlzc" <dl...@cox.net> escribió en el mensaje
news:b4fc260b-d6ef-4bdc...@googlegroups.com...

druzus

unread,
Sep 25, 2012, 1:51:18 PM9/25/12
to
On 24 Wrz, 15:38, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> Let's start with the simpler thing:
>
> FUNCTION MAIN()
>     LOCAL xVal
>     USE MYTABLE SHARED
>     WHILE !EOF()
>         xVal = FIELD -> myfield
>         SKIP
>     ENDDO
>     RETURN NIL

This code does not use indexes. The most expensive part of SKIP
operation
is index browsing. When index is not used then SKIP needs only one IO
operation: READ RECORD BODY

If index is active then it's necessary to make the following IO
operations:
1. LOCK INDEX FILE
2. READ INDEX SIGNATURE (to check if buffers should be discafred)
3. IF BUFFER WERE DISCARDED (OTHER STATION MAKE UPDATE AND
INDEX SIGNATURE HAS BEEN CHANGED) THEN READ INDEX NODES
FROM ROOT NODE TO LEAF NODE WITH CURRENT RECORD
4. GET NEXT RECORD NUMBER FROM BUFFER AND IF NECESARY
READ NEXT PAGE WITH LEAF NODE FOR NEXT RECORD
5. READ RECORD BODY
6. UNLOCK INDEX FILE

So in the best case at least 4 IO operations are necessary.
In the worst one 6 + index depth (number of pages from root
to leaf node).
If you enable SET DELETE ON or SET FILTER TO ... then
number of IO operations is of course bigger because they
are repeated for records which are out of scope.

Remote RDD can group all this operation in single IO operation
and this gives speed improvement.

> Will the above code get any benefits in using ADS?

No because it does not use index file so cannot be optimized by
consolidation of IO operation.
AFAIR ADS makes some group record reading and sending
anyhow it may change the order of IO operation and break
some user synchronization mechanism so it such optimizations
cannot be widely used in strict compatibility mode.

If you want to test network overhead then you can start with
HBNETIO - it's still not client server solution if you use it for
pure file access but it uses raw TCP communication so you
can see possible speed results with raw communication.
You can also check if it resolves the problem with Win7
and WS2008.
BTW I'm interesting in results.

Real remote RDD for sure speedups SKIPPING grouping IO
operations and evaluating filters on the server side (if possible).
But real speed improvement is moving whole application to
the server side. Such programs are even hundreds time faster
then the ones using ADS, MEDIATOR or any other remote RDDs.

best regards,
Przemek

fatfat

unread,
Sep 25, 2012, 2:00:26 PM9/25/12
to
therefore we are now using a telnet server to run my console xharbour
application from server which provide fastest speed for remote client :p:

"druzus" 在郵件張貼內容主旨
d51bf4f6-a658-445a...@o30g2000vbu.googlegroups.com 中撰寫...

fatfat

unread,
Sep 25, 2012, 2:08:21 PM9/25/12
to
both machine (all Server + Client) apply reg entry need for NT file sharing
same program with the following result :
no problem on Win7 (server) + Win 7 (client) with huge filtering, (got
acceptable display on skiping)
with problem on 2008 Server + Win 7 (client) with huge filtering (got a
noticable delay on skiping)

Thanks

"Patrick Mast" 在郵件張貼內容主旨
165b3c38-ce0c-46db...@googlegroups.com 中撰寫...

Hello,

> we have the same problem with 2008 server + Win 7 Client, but Win 7
> (shared)
> + Win 7 Client got acceptable speed : )

Still... The ONE main problem we have with our app is slow network access of
DBF files. Yes, we also tried TS and lately SQLRDD. But xHarbour should be
able to give us acceptable speed in a Windows network.

My 2cents

Patrick

Enrico Maria Giordano

unread,
Sep 25, 2012, 3:44:41 PM9/25/12
to
> This code does not use indexes.

Yes, my fault. The real test I'm using *do use* indexes but it's lost in
copy/paste operation. So I change the sample:

FUNCTION MAIN()

LOCAL xVal

USE MYTABLE SHARED

SET INDEX TO MYINDEX

WHILE !EOF()
xVal = FIELD -> myfield
SKIP
ENDDO

RETURN NIL

> If you want to test network overhead then you can start with
> HBNETIO

My friend Maurizio Ghirardini already tested it and told me that he get very
little speed improvement. Maurizio? Can you give us more details?

> But real speed improvement is moving whole application to
> the server side.

I agree. But unfortunately there are some serious problems in doing so. As
an example, what if an application uses a component (say, the email client
or the word processor)? It would run the remote one, not the local. Any
solutions? Or am I wrong?

Enrico Maria Giordano

unread,
Sep 25, 2012, 3:50:23 PM9/25/12
to
> no problem on Win7 (server) + Win 7 (client) with huge filtering, (got
> acceptable display on skiping)
> with problem on 2008 Server + Win 7 (client) with huge filtering (got a
> noticable delay on skiping)

Only a note: I don't expect that SET FILTER ON is fast even in a local PC
without a LAN.

druzus

unread,
Sep 25, 2012, 9:05:39 PM9/25/12
to
On 25 Wrz, 21:44, "Enrico Maria Giordano"

fatfat

unread,
Sep 25, 2012, 9:58:56 PM9/25/12
to
it work on my case (due to the data nature) with best result on local pc,
good result on win 7 (server) + win 7 (client ) , but a bit delay result
from 2008 server + win 7 client

Thanks

"Enrico Maria Giordano" 在郵件張貼內容主旨
50620aff$0$13283$4faf...@reader2.news.tin.it 中撰寫...

fatfat

unread,
Sep 25, 2012, 10:01:57 PM9/25/12
to
the telnet server that i use can shell to run remote side program within
telnet session : )


"Enrico Maria Giordano" 在郵件張貼內容主旨
506209a8$0$13278$4faf...@reader2.news.tin.it 中撰寫...

druzus

unread,
Sep 25, 2012, 10:14:07 PM9/25/12
to
On 25 Wrz, 21:44, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> > This code does not use indexes.
> Yes, my fault. The real test I'm using *do use* indexes but it's lost in
> copy/paste operation. So I change the sample:
[...]
If you need serious results then you should create self contain
example which creates tables, indexes and make some skip
operation with more or less aggressive filters.
In the past I posted some of such test code.

> > If you want to test network overhead then you can start with
> > HBNETIO
> My friend Maurizio Ghirardini already tested it and told me that he get very
> little speed improvement. Maurizio? Can you give us more details?

You should not expect noticeable speed difference because it's
physically
exactly the same number of IO operations. If you want to see something
like that then you can stop your investigation here. It's technically
impossible
because over 95% of time is consumed by IO operations if you access
files
by network. It doesn't matter what language you are using. Speed
improvement
can be only reach by reducing number of IO operation by moving to
other RDDs
which does not use network file IO operations or by introducing some
type
of read ahead buffers which looks nicely in tests but cannot be used
in
concurrent environment because they cause data corruption. Some of
such
read ahead features are supported by network drivers, i.e. the
infamous
opportunistic locks. It's very simple mechanism:
first user open file in pseudo shared mode - in practice use it like
in exclusive
mode reading data in very big peaces and buffering updates. It works
really
fast. In practice with comparable speed to files open in exclusive
mode.
When the second client open the same files then server should block
this operation for a while in which it sends information to the first
client
that it should send to the server all data written to file and
buffered on the
client side and then discard all its local caches because now
concurrent
access is enabled and due to other potential writers it's illegal to
use buffered
read data because it may not be updated. If all is well implemented
then
modified buffers are correctly save to files on the server and read
ahead
buffers are disabled so file access is safe for concurrent access but
suddenly
everything begins to work muuuuch slower. And this is expected
behavior.
You should be very concerned if you cannot observe such speed
reduction
because it means that opportunistic locks were not correctly disabled
and
so files cannot be safely used concurrently in RW mode by more then
one
client. It happens quite often in MS network drivers and it's the
reason why
we have to disable opportunistic locks in MS-Windows to eliminate file
corruption.
It's also known that in some cases even if you made modifications in
registry some operations are still buffered on the client side
improving
the speed results but with race condition which sooner or later causes
data corruption when two clients begin to update the same region using
it's own caches instead of real file body from the server. Due to
nature
of internal updates this race condition is the most danger for indexes
and usually they are corrupted first. Sometimes with small number of
updates such installations can work for without corruptions for few
days
or even week - they are lucky men - but of course anyone who seriously
think about his job must not make sth like that in production
environment.

I hope that now you have some basic knowledge about network driver
jobs and opportunistic locks. In fact there are also some more
complicated
improvements, different level of caches, special file access mode and
locks, etc. which can improve a little bit performance. Anyhow non of
them
can give you noticeable speed improvement with _SAFE_ concurrent file
IO RW access.

I asked you to make test with HBNETIO because it's very basic network
driver working on TCP connections dedicated for file IO operations so
it's overhead is minimal. Using it you could be also sure that non of
unsafe read ahead trick is still enabled in some MS network driver
layers. It means that HBNETIO results should show real network
performance - everything noticeable slower has some configuration
problems and should be reconfigured or other network drivers should
be used just like everything noticeable faster when more then one
client
open concurrently the same files in RW mode is not safe and sooner or
later will cause data corruption.
So you can check how close to HBNETIO your installations are.

If you need sth faster then you have to reduce number of IO
operations.
You can do that using dedicated remote RDD which does not operate
on files but sends complex request to server in single IO operation
and
server executes it giving the answer also in single IO (of course if
possible,
if you want to read record which is larger then data buffer in
ethernet frame
then it will have to be send in few frames though for upper level
protocols
like TCP it will look like single peace of data)

You can make some test with ADS, MEDIATOR or LetoDB. They
are classic remote RDDs. SQLRDD is also type of remote RDD though
unlike previous ones it does not use its own dedicated server but
tries to connect to RDBMS directly - it's a little bit less efficient
way
which introduce some limitation and incompatibilities to standard
Clipper/[x]Harbour/xbase++ RDDs behavior caused by RDBMS
client API.

As next step you can move your application to the server. If necessary
then you can divide it into client and server parts.

> > But real speed improvement is moving whole application to
> > the server side.
> I agree. But unfortunately there are some serious problems in doing so. As
> an example, what if an application uses a component (say, the email client
> or the word processor)? It would run the remote one, not the local. Any
> solutions? Or am I wrong?

RPC is the answer. You can use HBNETIO as RPC server not only
file server. BTW In Harbour SVN I committed some OLE server example
which allow to use HBNETIO RPC from any other language using OLE
interface.
For my own use I created GTNET library which allows to easy write own
terminal
servers with embedded RPC so my code is also on server and client
side.
I have full control of the GTNET client application from the server
accessing
local files, printers, COM ports just like the one on the server
without any
problem. And of course I use programs on client side, i.e. when I'm
presenting
some HTML reports then client automatically activates system www
browser.
The same is for product pictures, interactive e-mails, etc.
Of course locally open programs are not stored on the server side so
if user
turnoff his computer or run attach current session to other one then
only
the server application is switched to new connection.

best regards,
Przemek

Enrico Maria Giordano

unread,
Sep 26, 2012, 4:33:23 AM9/26/12
to
> You should not expect noticeable speed difference because it's
> physically
> exactly the same number of IO operations. If you want to see something
> like that then you can stop your investigation here. It's technically
> impossible
> because over 95% of time is consumed by IO operations if you access
> files by network.

Thank you. But let's start again from the beginning: in the '80, I remember
that Windows LAN was fast enough, no speed problem at all (and we had very
slow PC with very little RAM and 10 Mbps network cards). And no data
corruption. While the years passed away, I notice a constant decrease of the
Windows LAN speed, although we have powerful PC, tons of RAM and 1 Gbps
network cards now. How this can be explain?

> > I agree. But unfortunately there are some serious problems in doing so.
> > As
> > an example, what if an application uses a component (say, the email
> > client
> > or the word processor)? It would run the remote one, not the local. Any
> > solutions? Or am I wrong?
>
> RPC is the answer. You can use HBNETIO as RPC server not only
> file server.

Is it totally compatible with the already existing applications? I don't
want to rewrite them. :-)

Patrick Mast

unread,
Sep 26, 2012, 6:22:15 AM9/26/12
to
Hello,

> need to have NETRDD : )

We need a NON client/server solution ;-)

Patrick

Patrick Mast

unread,
Sep 26, 2012, 6:31:32 AM9/26/12
to
Hey Przemek,

> which does not use network file IO operations or by introducing some
> type
> of read ahead buffers which looks nicely in tests but cannot be used
> in
> concurrent environment because they cause data corruption. Some of
> such
> read ahead features are supported by network drivers, i.e. the

So, opening a network file in READ ONLY mode can be made FAST in a Windows network?

Patrick

druzus

unread,
Sep 26, 2012, 7:07:37 AM9/26/12
to
Yes it is. It's also configured in client/server settings.
It's possible to disable oplocks only for RW access and keep it
active for RO access. BTW AFAIR few times in the past users
reported here that they have performance problems with RW
access bit not with RO. This was the reason.
Anyhow it does not change the fact that such buffered RO
access is also not safe and can confuse RDD. I.e. it's
possible that you read one record and read ahead operation
read also half of next record buffering it on client side. Then
other station change partially read record. Finally you are
accessing this record and your network transport layer requests
rest of record from the server and you are presenting record
which is contains old and current data. The same may happen
with memo and index files. Depending on situation RDD may
not detect the problem and present corrupted data or can
generate corruption RTE, i.e. if reading next page from index
file it finds that it contains smaller keys instead of bigger
ones.
All such caches are danger. Anyhow if you accept the risk
then you can use it.

best regards,
Przemek

druzus

unread,
Sep 26, 2012, 7:53:08 AM9/26/12
to
On 26 Wrz, 10:33, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> > You should not expect noticeable speed difference because it's
> > physically
> > exactly the same number of IO operations. If you want to see something
> > like that then you can stop your investigation here. It's technically
> > impossible
> > because over 95% of time is consumed by IO operations if you access
> > files by network.
> Thank you. But let's start again from the beginning: in the '80, I remember
> that Windows LAN was fast enough, no speed problem at all (and we had very
> slow PC with very little RAM and 10 Mbps network cards). And no data
> corruption. While the years passed away, I notice a constant decrease of the
> Windows LAN speed, although we have powerful PC, tons of RAM and 1 Gbps
> network cards now. How this can be explain?

I also remember that when I was younger the world was most beautiful
and life easier ;-)
Fortunately now we can verify such things. I still have some really
old
installation which no one want to touch and I can verify it.
That's amassing how patient we were and what we could accept and
the performance of 10Mbs networks for sure is not acceptable in
current days.

Anyhow it does not mean that everything was worser. I.e. IPX/SPX
protocol is much more efficient in LAN then TCP/IP. It's always
faster.
For the same operation in LAN TCP/IP needs from few upto 50% more
time then IPX/SPX so effectively you need at least 20Mbs network to
be sure that you can keep the same speed when you switch to TCP/IP
from 10 Mbs IPX/SPX. BTW it was really funny to see in last years like
some idiots without any knowledge about low level network layer
implementations replaced local LAN IPX/SPX servers with TCP/IP
ones still working locally on the same hardware. And they even
suggested their customers that new solution is faster. There are
only two real reasons to change from IPX/SPX to TCP/IP
1. It's very hard and inefficient to use IPX/SPX in WAN
2. Problem with new software and hardware which does not longer
support IPX/SPX so TCP/IP is the only one choice.
But for sure it's not performance.
For LANs it's possible to write new custom protocol using UDP
which will use negative confirmation just like SPX to reduce
TCP overhead. Maybe I'll add such mode to HBNETIO in
some spare time. It should give next few small speed
improvement but it does not change the fact that it will
be still slow.

In last 25 years the speed of CPU and RAM has been increased
~10000 times but the speed of LAN only ~100 times.
so now the difference between local and network file IO access
is ~100 bigger then before. Now we have enough RAM to buffer
everything so local file access is not even reduced by hard disk
performance. In the '80 LANs were even faster then local hard
disk. So it's expected that you feel now it's slower but it's nothing
more then your personal imaginations which does not cover the
facts and real life tests.

> > > I agree. But unfortunately there are some serious problems in doing so.
> > > As
> > > an example, what if an application uses a component (say, the email
> > > client
> > > or the word processor)? It would run the remote one, not the local. Any
> > > solutions? Or am I wrong?
> > RPC is the answer. You can use HBNETIO as RPC server not only
> > file server.
> Is it totally compatible with the already existing applications? I don't
> want to rewrite them. :-)

Now you have exactly the same problem as other people who created
console application and still think about migration to GUI without
source code modifications. The answer is very similar.

If you want to reduce modifications in source code then you should
use remote RDD like ADS, MEDIATOR, LetoDB. With such RDDs
you only have to change in your code default RDD, sometimes add
some code to login to the server (it's not strictly necessary in some
cases) and precisely use filenames so RDD can detect if you access
local or remote files on the server. In some cases you can even
keep code which access the same files using network IO operations
and remote by RDD layer. Anyhow it reduces your choice to ADS and
LetoDB though AFAIR none of them allows automatic RDD switching.

Alternatively you can try to execute your application inside some
application server. Anyhow I cannot imagine that such general
Windows server can work effectively for bigger installations, i.e.
200 active hundreds clients.
And of course it still does not resolve the problem with access to
client station resources like files, printers, COM ports, local
applications, etc. Such application/terminal servers usually
gives some API to access them and you will have to adopt your
code to use it.

best regards,
Przemek

Ella Stern

unread,
Sep 26, 2012, 7:58:48 AM9/26/12
to
Sorry for jumping in once again.

Please do the following statistics, I mean comparing the complexity of your legacy application as it was 25 years ago, and as it is NOW

- the amount of raw data included in a database: total number of bytes occupied by the tables

- the complexity of a database depends on multiple factors: number of lines and colums for each table, number and length of index keys for each table, number of relations between tables

- the complexity of an application depends on the number of concurrent users, the number of concurrent write accesses on the same table, the % of records added/updated/deleted daily in the database, and the above mentioned amount of raw data, and the complexity of the database

SQL engines are SCALABLE client-server solutions respecting the data integrity in case of operations included in TRANSACTIONS. These two features are impossible to be achieved in case of free tables.

My two cents.

Ella

Enrico Maria Giordano

unread,
Sep 26, 2012, 9:01:24 AM9/26/12
to
> Please do the following statistics, I mean comparing the
> complexity of your legacy application as it was 25 years
> ago, and as it is NOW

There is not much difference. Table sizes and indexes key lenghts are
similar. And we have slow access even with one single user.

Reinaldo

unread,
Sep 26, 2012, 12:06:26 PM9/26/12
to
Hi everyone;

Moving to ADS should not only be easiest way to move to 2-tier cient-server paradigm -thus solving any access speed problems, it also adds so many benefits that I would overwhelm you if I tried to count them.

I don't think you have to change hardly anything -including your filters- to start taking advantage of ADS speed. Plus -ADS has AOFs that even faster!

1. SAP/Sybase is very interested in hearing from harbour/xharbour developers needing help on moving their apps over ADS. Let them know we exist. Access their newsgroup and ask, don't be afraid to tell what you need.
http://devzone.advantagedatabase.com/forum


2nd- I started a writing that should help any harbour/xharbour developer get started with ADS. Download from here:
http://ssfl.dynalias.com/temp_files/ads_from_xharbour.pdf

It probably has more info than you need, but it should help to get started.

3er- here is a .prg sample code that you can look at:
http://ssfl.dynalias.com/temp_files/createdd.prg

Hope that helps,


Reinaldo.

Reinaldo

unread,
Sep 26, 2012, 1:15:14 PM9/26/12
to
Fatfat;

Hi. ADS has AOFs which is exactly like Rushmore technology. I'm using very complex AOFs to filter only 10 records out of 1M using FW's xbrowse; the speed is amazing! Turn on or off the filter and see immediate response.

Using ADS is like using MSSQL or Oracle -only that you still get to have ISAM with an RDD. If you only have two users, then you could take advantage of the free "Full ADS server" which is free. Visit the devzone at www.devzone.advantagedatabase.com/forum and ask for the free 2 users server.



Reinaldo.

Enrico Maria Giordano

unread,
Sep 26, 2012, 1:28:53 PM9/26/12
to
> Moving to ADS [...]

I'd like to test ADS but, honestly, I'm not will to spend time on that
matter only to do a test. Is there a way to take an existing application,
add some line at the top (which line exactly?), include some LIB/DLL (which
exactly and in what order?), recompile the application and run it? If not,
I'm not going to proceed with ADS, I'm afraid.

Douglas Woodrow

unread,
Sep 26, 2012, 2:12:55 PM9/26/12
to
On Wed, 26 Sep 2012 19:28:53, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>> Moving to ADS [...]
>
>I'd like to test ADS but, honestly, I'm not will to spend time on that
>matter only to do a test. Is there a way to take an existing application,
>add some line at the top (which line exactly?),

Hi Enrico, have a look at the information Reinaldo provided, but I've
already shown you a simple working example:

//----------------------------------------
#Include ADS.CH

REQUEST ADS

Procedure Main()

InitADS( InsertYourDataPathHere )

// now open your tables as usual
[...]

Return

Static Procedure InitADS( cPath )
RddSetDefault("ADS")
AdsSetServerType( ADS_REMOTE_SERVER )
AdsSetFileType( ADS_CDX )
AdsConnect( cPath )
Return
//----------------------------------------


>include some LIB/DLL (which
>exactly and in what order?),

Here's my EXAMPLE.HBP file:
//----------------------------------------
-m
-n
-q
-gc0
-w
-es2
-lrddads
-lace32
-oAdsExample
EXAMPLE.PRG
//----------------------------------------

You'll also need the Advantage DLLs (ACE32.DLL and AXCWS32.DLL) and it
helps to have an ADS.INI file to speed up discovery of the Advantage
server.

>recompile the application and run it?

Yes, it really is as simple as that.

>If not,
>I'm not going to proceed with ADS, I'm afraid.

Przemek's excellent and detailed technical explanation will hopefully
have convinced anyone why it's a bad idea for a modern multi-user
database system to have multiple clients individually obtaining shared
access to the database.

ADS is a simple solution to this problem, and the enormous benefit is
that it requires hardly any changes to an existing Clipper/Harbour
application with a DBF-based database. (Incidentally, the default
communication protocol ADS uses is UDP, which has a slightly lower
overhead than TCP/IP).

--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 26, 2012, 4:32:52 PM9/26/12
to
> Hi Enrico, have a look at the information Reinaldo provided,

Too many pages. :-)

> but I've already shown you a simple working example:

Ok, I will follow your directions, step by step.

> #Include ADS.CH

First problem. I don't have ADS.CH. What I have to define to build xHarbour
with ADS?

Douglas Woodrow

unread,
Sep 26, 2012, 6:44:33 PM9/26/12
to
On Wed, 26 Sep 2012 22:32:52, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>
>First problem. I don't have ADS.CH. What I have to define to build xHarbour
>with ADS?

I'll leave that to the xHarbour guys to answer, but my advice would be
to use Harbour.

http://sourceforge.net/projects/harbour-project/files/

--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 27, 2012, 4:00:15 AM9/27/12
to
> I'll leave that to the xHarbour guys to answer, but my advice would be to
> use Harbour.

Sorry, I don't use Harbour and I have no intention to use it.

Patrick Mast

unread,
Sep 27, 2012, 4:21:31 AM9/27/12
to
Hey Enrico,

> Ok, I will follow your directions, step by step.
> > #Include ADS.CH
> First problem. I don't have ADS.CH. What I have to define to build xHarbour
> with ADS?

See c:\xHarbour\contrib\rdd_ads\

Patrick

Enrico Maria Giordano

unread,
Sep 27, 2012, 4:45:23 AM9/27/12
to
> See c:\xHarbour\contrib\rdd_ads\

Thank you.

Enrico Maria Giordano

unread,
Sep 27, 2012, 5:29:03 AM9/27/12
to
> REQUEST ADS

Ok, now I need ADS.LIB. How to build it?

I'm very grateful for your help.

Douglas Woodrow

unread,
Sep 27, 2012, 8:38:09 AM9/27/12
to
On Thu, 27 Sep 2012 11:29:03, Enrico Maria Giordano
<e.m.gi...@emagsoftware.it> wrote
>> REQUEST ADS
>
>Ok, now I need ADS.LIB. How to build it?

This is the batch file I use to build Harbour with ADS support:

// Build_Harbour_ADS.CMD ---------
set path=C:\compiler\mingw\bin;%PATH%
set HB_INSTALL_IMPLIB=yes
set HB_INSTALL_PREFIX=c:\HarbourADS10
set HB_WITH_ADS=c:\ads\acesdk\v10
win-make clean install
//-------------------------------------

(In case your resolution to avoid Harbour wavers :)

--
Doug Woodrow

Enrico Maria Giordano

unread,
Sep 27, 2012, 11:09:40 AM9/27/12
to
> >Ok, now I need ADS.LIB. How to build it?
>
> This is the batch file I use to build Harbour with ADS support:
>
> // Build_Harbour_ADS.CMD ---------
> set path=C:\compiler\mingw\bin;%PATH%
> set HB_INSTALL_IMPLIB=yes
> set HB_INSTALL_PREFIX=c:\HarbourADS10
> set HB_WITH_ADS=c:\ads\acesdk\v10
> win-make clean install
> //-------------------------------------
>
> (In case your resolution to avoid Harbour wavers :)

And for xHarbour? :-)

Patrick Mast

unread,
Sep 27, 2012, 11:41:40 AM9/27/12
to
Enrico,

> Ok, now I need ADS.LIB. How to build it?

You can use these with xBuild:
https://dl.dropbox.com/u/4973801/xBuild.bc5.ini
https://dl.dropbox.com/u/4973801/ads.lib.xbp

Patrick

dlzc

unread,
Sep 27, 2012, 12:27:00 PM9/27/12
to
Dear Enrico Maria Giordano:

On Thursday, September 27, 2012 8:09:41 AM UTC-7, Enrico Maria Giordano wrote:
> > >Ok, now I need ADS.LIB. How to build it?
>
> > This is the batch file I use to build
> > Harbour with ADS support:
>
> And for xHarbour? :-)

This is your thread, and you can let it devolve however you choose. But you were not interested in client / server solutions.

Przemek already pointed out, "terminal services" has the lowest network overhead of client / server, it requires no changes in your code base, with caveats that OLE and such runs on the server machine, and not your local machine. So client-side resources (printers, drives) need to be mapped for terminal service sessions, if required.

I found an additional thread that relates to classical client server, with the "AHA" moments down about 2/3rds of the way, December 2011:
http://social.technet.microsoft.com/Forums/en-US/w7itpronetworking/thread/96fe7d15-2bde-451b-8a12-796682a6cea8/

David A. Smith

Enrico Maria Giordano

unread,
Sep 27, 2012, 1:41:50 PM9/27/12
to
> This is your thread, and you can let it devolve however you choose.
> But you were not interested in client / server solutions.

You are right. I will create a new thread.

> Przemek already pointed out, "terminal services" has the lowest
> network overhead of client / server, it requires no changes in
> your code base, with caveats that OLE and such runs on the
> server machine, and not your local machine.

This "caveats" is not accepted by my customers... :-(

Enrico Maria Giordano

unread,
Sep 27, 2012, 1:42:34 PM9/27/12
to
I don't use xBuild, sorry.

Saulius

unread,
Oct 12, 2012, 6:07:31 AM10/12/12
to
Dear Enrico and ALL

I'm sorry I'm late... some homeworks done.
Can you and others take results from program provided in your networks,
both in Clipper (not below 5.2e) and xHarbour. Take info about network used.
Don't forget to set true patches.
TIA

//-----
#include "dbinfo.ch"

#define LOCAL_PATCH "c:\tmp" //<<<<<===== SET LOCAL PATCH
#define NETWORK_PATCH "i:\tmp" //<<<<<===== SET NETWORK PATCH

#define RECORDS_COUNT 30000
#define DBF_NAME "TESTONLY"

REQUEST DBFCDX

#ifdef __XHARBOUR__
REQUEST HB_GT_WIN
#endif

Proc Main()
Local nPatch, nAccess, nRec, cDbf, nStart
Field F1

? OS()
? Version()
? DTOC(Date()), Time()
? NetName(), Network(), nNetwork()

For nPatch := 1 to 2 //local/network
cDbf := if(nPatch == 1, LOCAL_PATCH, NETWORK_PATCH) +'\' +DBF_NAME
?
? cDbf, if(NetDisk(cDbf), "NETWORK", "LOCAL")

For nAccess := 1 to 2 //exclusive/shared
dbcreate( cDbf, { {"F1", "N", 10, 0} } )
USE (cDbf) VIA "DBFCDX" EXCLUSIVE
INDEX ON F1 TAG TAG1 TO (cDbf)
dbCloseArea()

dbUseArea( .t., "DBFCDX", cDbf,, nAccess == 2)
dbSetIndex(cDbf)
? if(DBINFO(DBI_SHARED), "SHARED ", "EXCLUSIVE")

nStart := Seconds()
For nRec := 1 to RECORDS_COUNT
dbAppend()
F1 := RANDOM()
next
?? Seconds() -nStart

dbGoTop()
nStart := Seconds()
While ! EOF()
dbSkip()
end
?? Seconds() -nStart

dbCloseArea()
fErase(cDbf +'.dbf')
fErase(cDbf +'.cdx')
next
next

Qout("<EOT>"); INKEY(0)
Return
//-----

Best regards,
Saulius


Enrico Maria Giordano

unread,
Oct 12, 2012, 5:06:31 PM10/12/12
to
> Dear Enrico and ALL
>
> I'm sorry I'm late... some homeworks done.
> Can you and others take results from program provided in your networks,
> both in Clipper (not below 5.2e) and xHarbour. Take info about network
> used.
> Don't forget to set true patches.

Here they are:

Windows XP Professional 5.01.2600 Service Pack 3
xHarbour build 1.2.3 Intl. (SimpLex) (Rev. 9744)
10/12/12 22:40:41
EMAG .T. .F.

e:\xharbour\tmp\TESTONLY LOCAL
EXCLUSIVE 2.94 0.02
SHARED 4.03 0.23

z:\tmp\TESTONLY NETWORK
EXCLUSIVE 3.05 0.03
SHARED 987.66 127.23
<EOT>

It's a wireless network (54 Mb).

Claudio Voskian

unread,
Oct 12, 2012, 6:50:29 PM10/12/12
to
Hi

If it matters, I'm using xharbour.com.

D:\TEMP>testprg.exe

Windows 7 Professional 6.01.7601 Service Pack 1
xHarbour build 1.2.1 Intl. (SimpLex) (Rev. 9427)
10/12/12 19:24:49
DELL-PC .T. .F.

c:\temp\TESTONLY LOCAL
EXCLUSIVE 1.75 0.02
SHARED 2.55 0.16

i:\temp\TESTONLY NETWORK
EXCLUSIVE 189.96 0.20
SHARED 515.91 70.35
<EOT>

The network is 100 mb ethernet.

Regards,
--
Claudio Voskian
Buenos Aires - Argentina
+54 911 4435-4789

"Saulius" escribió en el mensaje de
noticias:k58q52$u2v$1...@speranza.aioe.org...

Claudio Voskian

unread,
Oct 12, 2012, 7:52:34 PM10/12/12
to
Hi

Same computers and network, but running at 1 GBps instead:

D:\TEMP>testprg.exe

Windows 7 Professional 6.01.7601 Service Pack 1
xHarbour build 1.2.1 Intl. (SimpLex) (Rev. 9427)
10/12/12 20:44:42
DELL-PC .T. .F.

c:\temp\TESTONLY LOCAL
EXCLUSIVE 1.76 0.02
SHARED 2.52 0.17

i:\temp\TESTONLY NETWORK
EXCLUSIVE 104.08 0.15
SHARED 281.01 25.69
<EOT>

Regards,
Claudio


"Claudio Voskian" escribi� en el mensaje de
noticias:k5a6rk$b5k$1...@dont-email.me...

Hi

If it matters, I'm using xharbour.com.

D:\TEMP>testprg.exe

Windows 7 Professional 6.01.7601 Service Pack 1
xHarbour build 1.2.1 Intl. (SimpLex) (Rev. 9427)
10/12/12 19:24:49
DELL-PC .T. .F.

c:\temp\TESTONLY LOCAL
EXCLUSIVE 1.75 0.02
SHARED 2.55 0.16

i:\temp\TESTONLY NETWORK
EXCLUSIVE 189.96 0.20
SHARED 515.91 70.35
<EOT>

The network is 100 mb ethernet.

Regards,
--
Claudio Voskian
Buenos Aires - Argentina
+54 911 4435-4789

"Saulius" escribi� en el mensaje de

Enrico Maria Giordano

unread,
Oct 13, 2012, 5:37:28 AM10/13/12
to
> Here they are:
>
> Windows XP Professional 5.01.2600 Service Pack 3
> xHarbour build 1.2.3 Intl. (SimpLex) (Rev. 9744)
> 10/12/12 22:40:41
> EMAG .T. .F.
>
> e:\xharbour\tmp\TESTONLY LOCAL
> EXCLUSIVE 2.94 0.02
> SHARED 4.03 0.23
>
> z:\tmp\TESTONLY NETWORK
> EXCLUSIVE 3.05 0.03
> SHARED 987.66 127.23
> <EOT>
>
> It's a wireless network (54 Mb).

Now with an ethernet network (100 Mb):

Windows XP Professional 5.01.2600 Service Pack 3
xHarbour build 1.2.3 Intl. (SimpLex) (Rev. 9745)
10/13/12 11:29:42
EMAG .T. .F.

e:\xharbour\tmp\TESTONLY LOCAL
EXCLUSIVE 3.00 0.01
SHARED 4.03 0.23

z:\tmp\TESTONLY NETWORK
EXCLUSIVE 3.00 0.03
SHARED 309.30 39.56
<EOT>

mda.n...@gmail.com

unread,
Aug 1, 2018, 1:26:10 AM8/1/18
to
On Sunday, September 23, 2012 at 3:45:23 PM UTC+5:30, Patrick Mast wrote:
> Hello,
>
> > we have the same problem with 2008 server + Win 7 Client, but Win 7 (shared)
> > + Win 7 Client got acceptable speed : )
>
> Still... The ONE main problem we have with our app is slow network access of DBF files. Yes, we also tried TS and lately SQLRDD. But xHarbour should be able to give us acceptable speed in a Windows network.
>
> My 2cents
>
> Patrick

Hi Patrick,
i know this is very old thread. but i am having same issue right now.
i have dbase 5 + xhorbour application. when multiple user tries to access same db. it become very slow. i am hoping you can help me with this issue.

dlzc

unread,
Aug 1, 2018, 10:01:12 AM8/1/18
to
Dear mda.n...:

On Tuesday, July 31, 2018 at 10:26:10 PM UTC-7, mda.n...@gmail.com wrote:
...
> i know this is very old thread. but i am having same
> issue right now. i have dbase 5 + xhorbour application.
> when multiple user tries to access same db. it become
> very slow. i am hoping you can help me with this issue.

Please review this thread, for whatever insight you might gain.

Please START A NEW THREAD. It is unlikely that ANY solutions obtained in your case, will help the original problem reported 6 years ago.

A suitable title for the new thread might be "Network slows down, dBase5 and xHarbour, multiple users".

Then you'd need to tell us about the server OS, the OS on the dBase5 machines, and on the xHarbour machines.

Then you might want to tell us how many users you need, if it works faster with dBase5 only, xHarbour only, and maybe how many users before the network STARTS slowing down.

In general. Windoze has been going out of its way to make its performance numbers look at least as good as Linux, so they have been doing things with caches. And this has changed a LOT in 6 years, since this thread was started.

Immediate improvements can be had with changes in code base, by moving to SQL or other server-based solutions. Other improvements can be had by having terminal sessions (like "remote desktop", only lighter) running on the machine with the data, and you don;t have to change your code at all.

We cannot help you, with out more information from you.

David A. Smith
0 new messages