Performance of NetIO in WAN setting

[Ash]

Hello,

Test Environment:

Server located in Toronto

Internet Speed 38 Mb Down, 6 Mb Up

 

User 1 Local Workstation
Internet Speed 38 Mb Down, 6 Mb Up

Connected to server via WAN

 

User 2 in Oshawa (50 Km away)
Internet Speed 6Mb Down, 0.75 Mb Up

 

Both users are logged into the server during the test. Record locks are respected. The application has 48 tables and 35 indexes and opens all of them at startup.

 

Test results (All times are in seconds)

 

Function                                       User 2      User 1

Open all files (40 DBF, 35 CDX, 80 tags)       15          1                                
Browse paging (1 Page - 12 lines per page)     6           <1
Simple report (list, 1 page)                   4           <1

Report with data selection ½ page              35          <1

Report with data selection 7 pages             540         3

Customer balances report (4 pages)             70          1

 

User 2 has reported that the application is too slow for his liking.

 

These test results are from an application written back in 1986 and is now being converted to Harbour without significant code changes. Is there a way to improve the WAN performance?

 

Regards.

Ash

 

 

 

 

[Francesco Perillo]

Test results (All times are in seconds)

 

Function                                       User 2      User 1

Open all files (40 DBF, 35 CDX, 80 tags)       15          1                                
Browse paging (1 Page - 12 lines per page)     6           <1
Simple report (list, 1 page)                   4           <1

Report with data selection ½ page              35          <1

Report with data selection 7 pages             540         3

Customer balances report (4 pages)             70          1

 

User 2 has reported that the application is too slow for his liking.

User 2 was too generic... what is slow ? Everything ? Only the reports?

Anyway,  in a similar situation I use a VPN to log into the intranet where the server is and use remote desktop on a workstation that is local to the server.

 

These test results are from an application written back in 1986 and is now being converted to Harbour without significant code changes. Is there a way to improve the WAN performance?

As you said, the app is from 1986 and probably has some code that may be changed to use quicker algos.

Recently I sould see some very old code and it had several redundant code... code that was really quick when run once but made the program crawl when in a loop... or, in your case, when accessing the record takes long times over a wan

[elch]

Hello Ash,

the 5.th param of netio_Connect() is for compression level.

Not yet myself checked, as i need best speed in LAN, but maybe try:

HB_ZLIB_COMPRESSION_SPEED

As you certainly know, compress and decompress needs CPU power ...

And i'm unsure if this compression applies for all network action of HBNETIO.

Other values in hbzlib.ch

Please report if that changes something ..

best regards

Rolf

[Ash]

On Sunday, February 23, 2014 3:11:34 PM UTC-5, fperillo wrote:

User 2 was too generic... what is slow ? Everything ? Only the reports?

 

Almost all aspects of the application are slow: saving transactions, reporting, browsing, searching and so on.

 

 

Anyway,  in a similar situation I use a VPN to log into the intranet where the server is and use remote desktop on a workstation that is local to the server.

 

That would allow one RDP user, correct? I need a number of users accessing the server at the same time. Hence my move to NetIO.   

 

As you said, the app is from 1986 and probably has some code that may be changed to use quicker algos.

 

The application is amazingly fast in LAN environment.

 

Recently I sould see some very old code and it had several redundant code... code that was really quick when run once but made the program crawl when in a loop... or, in your case, when accessing the record takes long times over a wan

 

Fair point. I will review the code at some point with a view to improving its performance. The fact remains however that application is sluggish in a WAN setting.  I fear it might be something that I have not understood about NetIO.

 

[Ash]

Hello Rolf,

 

Tried the following code but the connection fails.

 

Regards.

Ash

 

IF ! netio_connect( alltrim( srvrname ), ;                       // Server name
                    iif( Empty( srvrport), 2941, srvrport ), ;   // Port
                    -1 , ;                                       // Timeout
                    "", ;                                        // Password
                    1 )                                          // Compression Level
   Alert( 'Unable to connect to server.' )
   QUIT
ENDIF

[elch]

Hi Ash,

hmmm ...

just checked, to get a connection with enabled 'compression' lead to no problem for me ..

.. but i am very quite unsure if compression is possible for work with DBF and not only for so called DataStreams ...

Because exact there i see the bottleneck for you:

6 Mega- BIT or BYTE ? internet connection, especially with an upload of only 0.75 x:

that is no question, very sure, *slowly* for database work over internet for your client 2 ..

best regards

Rolf

Tried the following code but the connection fails.

 

[Nenad Batocanin]

 

Anyway,  in a similar situation I use a VPN to log into the intranet where the server is and use remote desktop on a workstation that is local to the server.

 

That would allow one RDP user, correct? I need a number of users accessing the server at the same time. Hence my move to NetIO.   

 

Not necessarily. For example, TSPlus and WinFlector allow multiple connections on a regular (non-server) machine.

 

Regards, NB

[elch]

EXCUSE, Ash

you're right: i also can not connect with compression enabled ..

Management console reports a connect -- and immediate a disconnect.

But i am maybe on the track why not, HBNETIO seems must be recompiled for that ...

My successful first test with HB_ZLIB_COMPRESSION_SPEED does not change speed for browse in LAN,

at least nothing i can 'feel' ...

regards

Rolf

[elch]

again Ash,

yep, i can 'see', but not feel, a difference in LAN with compression enabled,

because the traffic is so much below the NIC capability.

If you want to quick test:

you use current branch from GIT ?, then:

line 222 in hbnetio.prg [ in line 221 you see: 'cPassword' ],

replace that upper 'NIL' ( there are two ) with 0 or -1 ...

*BUT* after this mod, you must use a compression value for netio_Connect(),

so a: HB_ZLIB_COMPRESSION: _DEFAULT or _NONE or _SIZE ...

as the default seem: .._DISABLE, which then leads to no connection.

best regards

Rolf

[Francesco Perillo]

 

That would allow one RDP user, correct? I need a number of users accessing the server at the same time. Hence my move to NetIO.  

Then change your program to run under linux and you can have hundreds of users all on the same server...

From another thread: "We were using Harbour for 4+ years on a terminal server with ~650 users (Debian Lenny)."

Here terminal server is not a MS thing, infact Lenny is the name aof a linux Debian distribution, dated 2009.

It all depends on what you need to achieve, on how many concurrent users you have...

[elch]

hello Ash,

you *also* got compression ready at work ?? - IMHO, i found the turning byte ..

Meanwhile i myself did some more tests.
As already said, in my develop environment, the *LAN* speed seem not to be the bottleneck:
in my unready test routines the NIC is perhaps max. 10% filled.
Internet (WAN) access is quite another topic of low network capabilties  ...

But one astonishing test results: i can trisecting network traffic with nearly
the same data rate result and only a handful more % CPU client workload.
As extreme, i watched 1/6 of network traffic, but this case also with 0.6 of max
data rate.

So the actual result below the line:
looks nice adjustable to our needs and constellation.

Please do not be confused by some fairytale with remote access of local working app:
that's maybe a solution for some special cases, and then can be mixed.

best regards
Rolf

[Ash]

Hello Rolf,

 

I got sidetracked with LetoDB.

 

My tests show that LetoDB is about 5-10 times faster than NetIO for the same application in a WAN setting. Now what makes LetoDB faster, I have not a clue. I have read a number of posts detailing technical pros and cons of these products but they are beyond me. NetIO is a solid product and performs flawlessly. LetoDB is just as good except for a few shortcomings outlined in this and other posts. It is hard to tell them apart in a LAN.

 

Why is it not possible for NetIO to achieve the same speed for the same application over the internet?

 

Regards.

Ash

[elch]

Hello Ash,

my tests showed an incredible fast DBSKIP() for LetoDB (*no* filter active) -- no idea why, that's a real master question.

There were in Windows some 'unpredictable' app crashes with LetoDB linked.

A check with valgrind in Linux showed multiple ! memory problems of the Linux app:

180B in 45blocks definitely, 33K!b in 145 blocks possibe lost; invalid read sizes; conditional jump on uninitialised value ...

-- only for just 20 seconds test run. And that was only the client lib, server itself i hadn't checked ...

So actual LetoDB in this state is no option for me.

---

If you don't get HBnetio compression to run for a check with your slow client, ask me ;-)

best regards
Rolf

[Przemyslaw Czerpak]

On Tue, 25 Feb 2014, elch wrote:

Hi,

> my tests showed an incredible fast DBSKIP() for LetoDB (*no* filter active)
> -- no idea why, that's a real master question.

There answer is trivial and you may find some of my messages about it.
LETO just like ADS asks about next or previous record sending only single
request and server with build in RDD makes the operation locally and
answers with record number and body usually in single IO. The same is for
SEEK operation.

Direct file IO access (also with HBNETIO used as FILE IO redirector) needs
much more operations, i.e. for SKIP with active index it needs:

LOCK INDEX FILE
READ INDEX HEADER WITH UPDATE COUNTER TO CHECK IF IT WAS CHANGED
IF INDEX WAS CHANGED SEEK CURRENT RECORD
* TAKE NEXT/PREV RECORD NUMBER FROM INDEX PAGE
READ RECORD BODY
IF RECORD IS NOT VISIBLE (DELETED OR FILTERED) GOTO (*)
UNLOCK INDEX FILE

Each (*) operation may cause additional IO request.
SEEK operation has in practice comparable cost to SKIP.

best regards,
Przemek

[Qatan]

Hello all,

 

I hope I am not doing something wrong on this nice list.

Please correct me, OK?

 

Microsoft created OneDrive. What is this (from Wikipedia):

 

OneDrive (previously SkyDrive, Windows Live SkyDrive and Windows Live Folders) is a file hosting service that allows users to upload and sync files to a cloud storage and then access them from a Web browser or their local device. It is part of the Windows Live range of online services and allows users to keep the files private, share them with contacts, or make the files public. Publicly shared files do not require a Microsoft account to access.

 

Well, I think it may be one more option for anyone and in case you are interested I just ask to use the link below to help me earn more free space...

 

Link: https://onedrive.live.com?invref=9265fbd6ad2eef49&invsrc=90

 

Thanks for any contribution!

 

Qatan

[elch]

Hi Qatan,

TL;DR on Wikipedia, only the key word was enough:
cloud

Hej, clouds will bring rain, which i do not really love when i am traveling outdoor -- and never ever trust fenced closed M$ ...

no tip, best regards
Rolf

[Qatan]

Hello Rolf,

 

> ... and never ever trust fenced closed M$ ...

 

I somehow agree and I will take care.

Thanks for the advise.

Regards,

 

Qatan

[Nenad Batocanin]

Ptsarenko fixed the some of the bugs that we reported. If you have precise information about a memory leak, send them to LetoDB forum.

 

Regards, NB

--
--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: http://groups.google.com/group/harbour-users
 
---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Nenad Batocanin]

Hello Przemek,

Do you maybe considering about adding some generic client-server function in NetIO? It seems to me that it is not too complicated, and it would be _extremely_ useful. It is enough to accelerate standard:

WHILE !Eof()
...
SKIP
END DO

Regards, NB